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Preface 


Since 1979, the Research and Division of the Association for Educational Communications and 
Technology (AECT) has sponsored or co-sponsored the publication of these Proceedings. Papers published 
in this year's 37th Annual Proceedings were presented at the 2014 AECT Convention in Jacksonville, FL.A 
limited quantity of these Proceedings were printed and sold in both hardcopy and electronic versions. 
Volumes 1 and 2 are available through the Educational Resources Clearinghouse (ERIC) System. 
Proceedings volumes are available to members at AECT.ORG. 

The Proceedings of AECT’s Convention are published in two volumes. Volume #1 contains papers dealing 
primarily with research and development topics. Papers dealing with the practice of instructional 
technology including instruction and training issues are contained in Volume #2. This year, both volumes 
are included in one document. 

REFEREEING PROCESS: Papers selected for presentation at the AECT Convention and included in these 
Proceedings were subjected to a reviewing process. All references to authorship were removed from 
proposals before they were submitted to referees for review. Approximately sixty percent of the 
manuscripts submitted for consideration were selected for presentation at the convention and for 
publication in these Proceedings. The papers contained in this document represent some of the most current 
thinking in educational communications and technology. 


Michael R. Simonson 
Editor 
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Information Visualization in Students Eye: 
An Eye Tracking Study of Rising Sea Levels 

Dalia Alyahya Suzan Alyahya 


Keywords: eye traeing, information visualization, multimedia, eognitive proeesses, visual instmetion, multimedia 

learning theory 

Abstract 

This presentation foeuses on how learners’ proeess information visualization by exploring the relationship 
between fifty undergraduate learners’ performanee and their viewing behaviors gained from eye traeking. Students’ 
attitudes towards learning by using graphies, along with quantitative and qualitative results have been diseussed. The 
results revealed a deeper understanding of how learners proeess visual information, and therefore assisted 
instmetional designers in ereating more effeetive visuals. 

Background 

Endless benefits are yielded through evaluating data that has been transformed into visual representations. 
Shepard (1967) found that the storage of visual information was far superior to that of verbal information. Visual 
instruetions faeilitate and transform learning by eommunieating larger amounts of information more quiekly than 
text, while showing relationships among meanings and other various faetors (Tufte, 1990). 

Learners’ use of eomplex visual information still needs to be investigated. Eye traeking teehniques eould 
provide a deeper insight into how eomplex information is proeessed by an observing eye movement and the 
distribution of visual attention, sueh as: where partieipants are looking (eye fixation), how long they foeus on an 
objeet (fixation duration), and in what sequenee they view information (viewing path) (Holsanova et ak, 201 1). 

Riehard E. Mayer has extensively researehed hypotheses about the pereeptual proeess used while learning 
from multimedia materials (Mayer 2010). He stated that, “eye traeking measures, sueh as total fixation time on 
relevant areas of an instmetional graphie, ean be sueeessfully added to researehers’ toolboxes as a way of testing 
hypothesis about pereeptual proeessing during learning under different instmetional methods,” (Mayer, 2010, p. 

169). In addition, eye traeking is a very valuable tool, as it helps to identify the information that is being proeessed 
quiekly and automatieally (Huber and Krist 2004; Seheiter & van Gog, 2009). 

Sinee minimal researeh has been done on multimedia learning and eye traeking teehnology, several authors 
have suggested additional exploration of the topie to gain further understanding (Ainsworth, 2006; de Koning, 
Tabbers, Rikers, & Paas, 2009; Liu & Chuang, 2011; Sehmidt-Weigand & Seheiter, 201 1). Only a handful of studies 
have utilized alternatives to test seores as a means of studying the effeet of visuals on learning (Liu & Chuang, 

2011), and few have examined the proeess of information visualization of eomplex graphies without text. Moreover, 
Tufte’s assumptions regarding the effeetiveness of information visualization (Tufte, 1990, 2001) are not researehed 
based, whieh make it worthy of additional researeh. This researeh foeused on examining the learners’ eognitive 
proeesses when viewing information and its relationship to their performanee. 

Research Questions 

The purpose of this study was finding the relationship between learners’ performanee and eye fixation 
measures overall and for speeifie Area Of Interest (AOI). Also, this study seeks to ereate a more eomprehensive 
understanding of how do learners view visual information and what is there attitudes toward graphies as a souree of 
information. 


Method 

This study followed a mixed method design to obtain quantitative and qualitative answers to the researeh 
questions. The researehers employed multiple regression analysis to examine how eye fixation measures influeneed 
learners’ performanee, and triangulation of data sourees to validate the quantitative results. The dependent variable 
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here was learners’ performance and the independent variables were the eye fixation measures which are fixation 
duration, number of fixation, and number of visits to a specific AOI. Eye -tracking visualization tools like heat maps 
and eye paths were used to examine how learners’ view the visual information. Finally, personal interviews were 
conducted to explore the learner’s attitude towards obtaining information from graphical content. 

Subject: 

This study was conducted at a university in the central region of Saudi Arabia, with a convenience sample 
of approximately fifty learners. Ten students were randomly selected for interviews from the total fifty learners. 

Material 

The researchers used the, “Rising Sea Levels,” graph from McCandless’s book (2009) to conduct the study. 
This graph predicted the flood impact upon cities in the event of an increase in sea level. 

Procedure 

Learners were approached in public areas on campus with minimal disruption. They were asked to 
participate for approximately ten minutes. They were given a cover letter and consent form, and after their approval, 
they began an eye-tacking calibration, and started the task. Upon completion of the task, they took a test. After the 
test, learners were randomly selected for an interview. 


Results 

The graph appeared on (Figure 1). Table 1 shows the descriptive statistics for graph fixations. 

Table 1 Descriptive Statistics for the variables 


Items 

n 

Mean 

Std. Deviation 

Performance 

50 

2.180 

1.304 

Number of Fixation on AOI (counI) 

50 

116.100 

89.463 

Fixation Duration on AOI (in secs.) 

50 

0.355 

0.079 

Number of Fixation of whole graph (counts) 

50 

295.240 

217.198 

Fixation Duration of whole graph (in secs.) 

50 

0.353 

0.079 

Number of Visits to AOI (counIs) 

50 

22.000 

15.802 


The results shows that there was no significant correlation between Performance and Fixation Duration of 
the Whole Graph at significance level , R= 0.015 , t statistics = 0.101 , and P-value = 0.920 > 0.05. Flowever, there 
was a correlation between performance and the number of fixation on the whole graph at the significance level , R= 
0.291, t statistics = 2.105, and P-value = 0.041 < 0.05. Number of fixation and fixation duration for the AOI showed 
no relationship with the learners’ performance. Flowever the correlation between number of visits on the AOI and 
performance resulted in a significant correlation with R=0.276, t statistics = 2.204, and P-value = 0.042 < 0.05. The 
two correlations were consistent with those from a number of earlier studies. Flowever, those earlier studies showed 
stronger relations. (Liu & Chuang, 2011; Rayner, Yang, Schuett, & Slattery, 2013; Schmidt- Weigand, & Scheiter, 
2011; Yang et ah, 2013). It was assumed that the graph was easy to understand and learners do not need longer time 
to process information “Longer fixations are generally believed to be an indicator of a participant’s difficulty in 
extracting the information from a display” (Jacob & Kam, 2003, p.585). Significant relationships were found 
between the performance and number of visits (p-value 0,042 0.050). When learners make more visits to the central 
area in the graph, there performance increase. Figure 1 shows the eye movement of learners, while comparing how 
the level of the sea changed through the years while going back and forth between the maps. The finding revealed 
that the time of fixation duration couldn’t be predicted from the learners’ performance. An explanation could be that 
the given graph was not complicated and required less processing time. Also, performance test measures specific 
number, which was hard to recall due to the high load of information visualization. 
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Figure 1 shows sample of scan path of the four participants view. The sample path supported 
comprehensions of individual participant’s behavior through the plotted starting points, fixation location, and 
durational indicator. The learners viewed the picture as a whole and studied the picture’s details. The graph shows 
that the learners spent more time on the right side of the picture due to the amount of information. 


Figure 1. Sample of scan path of four participants view 



In figure 2 the sample of scan path of one participant’s shows that the patters of view was in two main 
directions, horizontal and vertical. This was due to the figure design of the information being arranged in the graph 
edges. This figure shows that the learners use compare and contrast to make meaning of the graph. This can be seen 
in the horizontal and vertical lines. Some variables needs more time to understand than others. This can be seen in 
the verity of circles size. 
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Figure 2. Sample of scan path of a participant view 



Figure 3 shows the heat map of the participants view. Fleat map indicated the highest viewed area by 
distinguishing it in a distinct color which is on the right side of the sea level graph. As Jacob and Kam (2003) stated, 
“the number of fixations on a particular display element (of interest to the design team) should reflect the 
importance of that elemenf’ (Jacob & Kam, 2003, p. 585). In Figure 3, the colors Red, Yellow and Green represents 
the most, normal and least important areas viewed by the participants respectively. 


Figure 3. The heat map of the participants view 
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Figure 4 shows a comparison between the highest five grades and the lowest five grade students heat maps. 
Fleat maps shows a higher attention on different information for the higher grade learners than the lower grade 
learners. Furthermore, higher grade learners has less attention on the middle part of the map that has less 
information than the lower attention learners. This is consists with what Rayner (1998) points out that longer 
fixation duration are generally indicative of more extensive processing which does not correspond to the current 
study. 



Fi^re 4. A heat map comparison between highest and lowest students performance. 


High Performance 


Low Performance 


Moreover, Leaners found the Rising Sea Levels graph interesting and flin for studying. Also, learners’ 
attitude indicated that they found graphics helpful in understanding the information “I understand the graph quickly 
and easily”, “I like to see the information visually, I can remember it for a longer period of time”. Also learners 
thought that the graph helped them to remember the information and recall it easily, “I do not remember every single 
city, but I got the main idea about how sea level affects the earth”, “I can remember some of the main cities like 
New York”, “and the graph is there in my mind, I can recall if’. However some learners indicated that the concept is 
complicated, “it is hard to remember everything, and there is so much information”. This concludes that the learners 
form two levels of understanding, the conceptual idea and the specific information such as the city name. Moreover, 
the graph has helped the learners to recall information and easily understand the concept. 

This study is focused upon information visualization graph with complexity in order to measure learners’ 
performance and explore their attitude. However it is recommended to conduct further qualitative studies based 
upon multiple images with different levels of complexities. It will provide more detailed understanding of cognitive 
multimedia learning, learner experiences and learner perceptions. Furthermore, additional studies can be carried out 
to measure analytical abilities through visual information of groups based upon gender, IQ (Intelligence Quotient) 
and educational qualifications. 

The possibilities of research studies with these instruments are unlimited. Studies that explores cognitive 
activities on Complex charts, graphs and maps, differences between general images, images with voice overs and 
images with interactivity will be beneficial for the designers and developers to prepare more appropriate graphical 
content. Further studies needed related to multimedia learning based on color coding while designing instructional 
material and diverting learners’ attentions to a defined area of interest. It will provide firm background and 
guidelines for the instructional designers and developers to develop effective multimedia enriched pedagogical 
agents that communicate efficiently with the learners. 
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Interactive Ebooks as a Tool of Mobile Learning for Digital-Natives in Higher 
Education: Interactivity, Preferences, and Ownership 


Aadil Askar 

King Sand University 

Abstract 

As usage of mobile technologies for academic learning became popular, it is important to investigate the 
advanced features of interactive eBooks to develop effective interactive learning content. It will ultimately introduce 
instructional designers and eBook developers to relate interactive eBook features to its cognitively relevant 
characteristics. It is also important to identify the student learning preferences as learner characteristics to help 
instructional designers properly implement eBook and target their audience. This study investigated the eBook 
learning preferences of the student community at King Saud University by conducting surveys throughout the 
Preparatory Year Campuses. In this paper the data from 387 valid responses were collected and analyzed using 
SPSS22 software, and the results of student survey are shared, which provide insight factors such as availability of 
appropriate framework for implementation of interactive eBooks and learner preferences for interactive eBook 
features. 


Introduction 

The time frame characterized by the existence of digital technology is known as Digital Era and Digital 
Natives are people who were born after the extensive usage and adoption of digital technologies. It doesn’t refer to a 
particular generation but to the children who have grown up using these technologies like Internet, Computers, and 
Smart-mobiles. Although the advancement in technology has facilitated today’s learners to use various types of 
mobile tools for learning throughout the globe, the student community is the major user of the latest mobile 
technologies across Saudi Arabia. As digital natives, they follow the modem social media trends and are likely to 
have different expectations and behaviors towards the use of mobile technologies for learning. Al-Fahad (2009) 
noted that students at King Saud University, Saudi Arabia, demonstrate positive attitude and perceptions towards 
mobile learning. He described that “They changed from passive learners to tmly engaged learners who are 
behaviorally, intellectually and emotionally involved in their learning tasks” (p. 118). 

Currently, pioneering technologies like interactive eBooks together with Smartphone & tablet devices are 
the essential tools for m-Leaming. The development of eBook content optimized for mobile devices can result in 
better learning outcomes. One popular technique is to include interactive features that will keep learners motivated 
and engaged throughout the learning process. Most of the time, this involves adding interactive elements like 
graphical content, audio-visual content, animations, interactive infographics, online video streaming, interactive 3D 
elements, live polls, quizzes, and multiple types of questions. According to Zajac (2009), letting learners identify 
their own learning style, and then giving them an opportunity to match activities to their preferences will help them 
significantly, as they are individuals and hold their own learning preferences, strengths and skill sets. 

Many software tools including HTML5, enhanced interactive features of eBooks to its highest extent. With 
the advancement of technology in mobile devices such as Smartphones and Tablets, it is now possible to embed 
these features into eBooks. Currently, it’s a trend among individuals to own high-end mobile devices and use most 
of its prominent features. Most people are aware of these technologies and students are no exception. As (Figure 1) 
shows. Global Web Index (2013) report states, Saudi Arabia ranked first in the highest twitter using countries. 
Mobile devices are highly adopted in Saudi Arabia, affecting the increased use of mobile applications such as twitter 
& Facebook. 
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Figure 1\ Countries Ranked in Order of the % of Twitter users in their Internet Populations 


Higher education in Saudi Arabia aims toward developing and adopting m-Leaming in academic delivery 
(KSU Library & NCEL, 2014). Interactive eBook is one of the on-demand m-Leaming solution which has greater 
flexibility and accessibility than paper-based text. The multimedia-enriched visual appeal has the potential to 
integrate various supportive materials for personalized learning. Still, it’s not been completely adopted by 
institutions as essential learning tools. It is important to have a full-fledged framework for the development of 
interactive eBooks, which offers exceptional features like interactive multimedia, assessments, progress reporting 
etc. Strategies like Gamification and Storytelling can also be integrated into Interactive eBooks to revolutionize the 
eBook learning environment. 

Furthermore, in order to help instructional designers and eBook developers to implement effective 
interactive learning content, it is essential to investigate the importance of eBook features and how it can enhance 
learning, along with the availability of appropriate framework, learning preferences as learner characteristics while 
using interactive features of eBooks. The term “Appropriate Framework” refers to the student device ownership and 
the knowledge of using technology and its supporting infrastructure like Computers, Internet, and Smartphones etc. 
To accomplish successful implementation of interactive eBooks, it’s important to understand the present learning 
environment and identify the level of intervention required. 


Literature Review: 

The present eBook development is aimed primarily to digitize printed works rather than to develop and use 
the new technology to support learning (Coyle 2008). Hence, many researchers (Berg, Hoffmann & Dawson, 2010; 
Bierman, Ortega & Rupp-Serrano, 2010; P. Lam, S. L. Lam, J. Lam, & McNaught, 2009; Pattuelli, & Rabina, 2010; 
Woody, Daniel, & Baker, 2010) have investigated to integrate eBooks into academic learning. Generally users 
prefer to interact with resources and have functionality to which they have become familiarized in their eBook 
experiences (Biemian et al. 2010). However, according to the results of these studies both teachers and students 
believe that current eBooks are not ready to be used as printed books. Unlike printed books, the eBooks should be 
developed using a more constructive design to build virtuous user experience (Woody et al. 2010). The concerns of 
usability and functionality therefore became essential for wider adoption of eBooks, especially in academics (Berg 
etal. 2010). 
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According to Huang, Liang, Su and Chen (2012), the response from 166 elementary school students shows 
that the eBook has more preference than printed book in terms of usability and functionality. Korat (2010) 
conducted many studies on children with eBook reading and emergent literacy which includes vocabulary, word 
recognition and phonological awareness. The results shows that all the students’ vocabulary, word recognition and 
phonological awareness were improved by reading eBooks. (Korat 2010; Korat & Shamir 2007, 2008; Segal-Drori, 
Korat, Shamir & Klein, 2010; Shamir, Korat & Barbi, 2008). Grimshaw, Dungworth, McKnight, and Morris (2007) 
found that multimedia enriched content in eBooks, such as narration along with animated pictures and sound effects 
shows improvement in children’s reading. 

Haiguang, Chenzhu, Pan and BaoCong (2012) says “Learners are acquiring more knowledge than 
before and that requires flexible learning method to limit time and improve efficiency. Mobile learning is 
which use fragmented time to learn in a self-regulated way. Learners will not passively accept information 
from others, but to choose learning content according to their own needs and learning goals.” (p. 1337). 

Students are using mobile devices in their personal lives in ever-increasing, creative ways. They will begin 
demanding learning and productivity applications at colleges, if they haven’t already. 

Research Questions 

This study aims to identify the students learning preferences as learner characteristics to suggest the 
importance of interactive eBooks in Higher education while demonstrating the existing framework, prior 
knowledge of its usability and device ownership. In order to achieve the above described objective, the 
Researcher has conducted this study to answer the following research questions. 

1 Are students reading and interacting significantly more with Social media enabled mobile devices 
than before, as interactive features became handy? 

2 What are the student preferences of interactive eBook features? 

3 Does the student ownership of the mobile technology form an appropriate framework for 
implementing m-leaming environment? 

4 Which of the eBook features are capable of resembling a face-to-face learning activity from 
students’ point of view? 

Participants: 

The survey was distributed electronically to the Preparatory Year students at King Sand University. The 
number of students at PY, KSU are approximately 12000 students distributed among 5 Campuses. The survey was 
developed & distributed to participants via University email ID through an online survey system. The initial targeted 
population was 12000 students, but due to the exam preparation constraints a very less number of responses were 
experienced. Therefore 387 complete responses have been used. To ensure the privacy of the participants, the survey 
was conducted anonymously. 

Materials: 

The eBook features survey was developed and derived from the literature as well as eBook features 
supported by eBook development software. Even though the participants may or may not have ever used an 
Interactive eBook, the researcher asked them to study each feature and rank its importance and whether they are 
resembling the in-class face to face learning activity. Although these two are different in nature, the Researcher’s 
aims to enquire about student perspectives on the similar activities that takes place in both environments. 

The survey item includes a 5 point Likert scale questions that ranges from “strongly disagree” to “strongly 
agree” which holds lowest rank to highest rank respectively. For example, “I prefer to use eBook exercise 
activities”. Also, “yes/no” type questions were used. For example “Do you have a smart device?”. The complete 
survey items were developed in Arabic Language, as it is the primary and official language of Saudi Arabia. 

Procedure: 

A survey was developed by reviewing the literature and eBooks development software tools and its 
features. Then it was submitted to a review panel of three professionals in the field of educational technology. A 
Cronbach’s alpha was conducted for reliability testing. During the fall of 2014 semester the survey was distributed 
to approximately 12000 enrolled students via online survey system. Although the target population was 12000 
students, the survey was distributed during Final Exam preparation week, where less number of students access their 
University Emails which is the only delivery method used. Thus the responses collected were approximately 700. 
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The survey was developed and distributed using online survey system. After condueting the survey the data was 
downloaded and analyzed using SPSS statistical software. 

Results & Discussion 

The survey was conducted using electronic survey system and out of approximately 700 responses from 
Preparatory Year students, 387 valid responses were used for analyzing the survey data. The analysis was done using 
SPSS statistical analysis software. To identify existing framework, learner preferences, and device ownership the 
frequency analysis has been used. 

Framework. In order to investigate the prior knowledge and readiness of the infrastructure for implementing e-leaming 
the researcher asked questions related to device ownership and usage. The data indicated that, 100% of the Preparatory 
Year students use Smart devices along with Social Media apps like Twitter, Facebook and WhatsApp, out of which 
88.6% of them uses either Android or iOS based Smart devices which are prominent e-leaming platforms. Rest of the 
11.4% uses Blackberry, Windows and other Operating Systems as their mobile platforms. This high percentage of 
Smart device ownership facilitates recurrent usage of mobile Social Media apps which adds support to earlier finding 
of Global Web Index (2013) report indicating Saudi Arabia being ranked first among Twitter using countries. 94.3% 
of the sample have been using these devices for more than one year and a total of 90.4% already uses it for academic 
purposes. Internet availability is wide spread throughout Saudi Arabia with its leading mobile operators. Thus 100% 
of the students have access to public or private Internet, and 94.8% of the sample reported that they have private 
Internet service. 

Learner Preferences'. Although students may or may not have used interactive eBook earlier, the researcher asked 
them to provide their perceptions about the interactive eBook features which were described by their definitions. 


Interactive eBook feature (N=387) 

Rank 

Percentage 

Bookmark: Bookmarks stores page or section information of the eBook for future 
retrieval. 

1 

93 

Local and web search: Users can search for information within the eBook, or search over 
internet. 

2 

92 

Table of Content: It is an Index of the eBook that allow the user to open a specific 
chapter or topic. 

3 

89 

Portable: The user can download it and easily carry it anywhere anytime with-in their 
Smart devices. 

4 

89 

Interactive Images: It is a graphical image that provides additional information of its 
parts (areas) upon the user interaction. 

5 

89 

Multimedia: Audio visual content or Multimedia based content, in an eBook allows user 
to watch videos or animation to learn topics effectively. 

6 

89 

Highlighting and Note taking: The user can highlight the content or take notes which 
will be available as Study Cards for easy one point access for future reviews. 

7 

89 

Content Sharing: Sharing allows users to exchange knowledge of an eBook through 
Social Networks. 

8 

89 

Image Gallery: It is a set of related Graphical Images of a certain topic. 

9 

88 

Interactive 3D virtual graphics: It is a Graphic image virtually demonstrated in 3D. It 
can be viewed in multiple angles upon the user interaction. Example: DNA, Atomic 
models. Chemical Structures etc. 

10 

87 

Word Wrap: It allows the eBook to rearrange its content according to the device 
orientation and screen size. 

11 

87 

Assessment: User can assess and evaluate their learning process by using interactive 
objects such as multiple choice questions, matching, identifying an object in an Image and 
so forth. 

12 

85 

Dictionary Integration: It displays dictionary reference of a selected word with-in the 
eBook. 

13 

84 

Flexibility: eBooks are flexible enough of Updating & Appending the learning content. 

14 

84 
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Interactive: Provides interactive learning objects such as games, image galleries, 3D 
Graphic images, videos, assessment tools, and web forms which allows users to interact 
and exchange information with the eBook. 

15 

84 

Text to Speech: It allows users to listen to the selected text through the device speakers. 

16 

83 

Simulator: Simulators in eBooks recreates real time environment to provide hands-on 
experience for Software, Anatomy, Surgery and astronomy etc. 

17 

82 

Feedback: User can participate in online feedbacks, surveys in the eBooks, which will 
submit their views, experiences and suggestions to the development team 

18 

80 


Table T. Learner preference about eBook features. 
Note: The survey has these definitions in Arabic Language. 


The analysis of this study revealed that the least preferred feature is Feedback and it holds an 80% of 
preference, where the most preferred feature is Bookmark which holds 93% of preference. The variation between 
the most preferred and least preferred features is about 13%. This variations implies that students being digital 
natives have the knowledge and skills to use these features. Flowever, it has been observed that the generic eBook 
features (without much interactivity) are most preferred by students, and most recent intuitive features are less 
preferred. It may be due to the lack of prior knowledge of their usability, feeling attached to it and their lack of 
understanding in their offerings. Thus features like Bookmarks, Searching and Table of contents became most 
preferable which are very common in every format of eBooks regardless of interactive or not. And features like 
Assessments, Text to speech. Simulators and Feedback became less preferable which has potential to revolutionize 
the concept of eBooks and bring them to an advance level. 


Learner Preferences about eBook features 



#■ 


Feature Name 

Figure 2: Learner preferences about eBook features 


The advancements in technology leads to the development of tools to perform human like activities. This 
motivated the researcher to investigate, how far these tools are effective, how efficiently they can be used and 
learners observations about these features, when compared to the face-to-face learning in order to facilitate the 
development to match the learner characteristics. Following table describes the outcome of the survey items that are 
targeted to compare the resemblance between interactive eBook feature and face-to-face learning activity. These 
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items are eolleeted on a 5 point Likert seale in whieh “Strongly Agree” has the highest degree and “Strongly 
Disagree” has the lowest degree of preferenee, and are ranked aeeording to the Agreement pereentage. 


Rank 

Activity Comparisons 

Strongly 

Disagree 

% 

Disagree 

% 

Neutral 

% 

Agree 

% 

Strongly 
Agree % 

1 

It is convenient to carry Portable 
interactive eBooks in one device? 

1.3 

5.4 9.0 

28.4 

55.8 


2 

I can learn from replaying the Audio Visual 
content of Interactive eBooks as classroom 
lectures? 

1.6 

7.5 

9.6 

33.3 

48.1 

3 

Interactive 3d, Image Gallery, Images gives 
me in-depth knowledge of graphical 
instruction as the Graphics and drawings 
used in the classroom? 

1.6 

4.4 

9.6 

38.5 

46.0 

4 

The Quality of the information of Web 
search feature is as same as the classroom 
infonuation? 

2.3 

8.3 

13.4 

35.4 

40.6 

5 

Instant results and other tools of Interactive 
eBook Assessment helps me evaluate 
myself accurately, as effective as the 
traditional classroom assessment and 
feedback? 

1.6 

12.1 

23.0 

27.4 

35.9 

6 

Sharing the Interactive eBook content over 
social media provides me reviews and 
suggestions as the classroom discussions? 

2.8 

11.1 

14.5 

38.0 

33.6 

7 

The Simulators of Interactive eBooks 
provides me practical knowledge as the 
classrooms activities? 

2.8 

11.9 

18.1 

35.1 

32.0 

8 

Highlighting and Note taking (Study 
Cards) in Interactive eBooks is as easier as 
taking class notes? 

5.9 

26.9 

15.5 

26.4 

25.3 


Table 2: Learner pereeptions about similarities between eBook features and in elass aetivities. 
Note: These eompared aetivities are extraeted and translated to English language, as the original language 
used was Arable. 


Device ownership'. The analysis of this study also shows that approximately 90% of the partieipants are using either 
Android or iOS based mobile deviees. It is very important for the interaetive eBook developers to foeus upon the 
platforms they are targeting for. Generally these interaetive eBooks are paekaged in ePub format whieh is widely 
used by most of the platforms baeked by their deviees. But to utilize the strength of a platform or deviee, these 
eBooks ean be fine-tuned to perform at its best. It’s also known that some of the platforms doesn’t support some of 
the very important features, for example, Apple’s proprietary eBook development tool doesn’t fully support Arable 
language. In these situations the developer ean go for the development of third-party tools to integrate these features 
into the platform. This study also shows that 100% of the partieipants has aecess to publie or private internet. This is 
helpful for the developer to proeeed without worry to the development of more internet based features of interaetive 
eBooks. For example, features like Feedbaek & Progress Reporting, Assessments and Interaetive Animations ean be 
developed whieh make the eBooks more interaetive and outstanding. 

Conclusion 

This study is signifieant as it examines the interaetive eBook features whieh are rarely explored, and also 
aeknowledges the available landseape and the gaps to form a baseline to move forward. In this study the researcher 
has investigated the learner preferences, device ownership and available framework. He found that learners gave 
more preference to the features they already know. Most of the latest and far more advances features are less 
preferred as they are new and unfamiliar. This supports Zajac’s (2009) statement saying, learners are individuals 
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who hold their own learning preferenees, strengths and skill sets. Letting them identify their own learning style and 
match their activities to their preferences will help them significantly. Thus it is suggested that, conducting this 
study with proper training of the usage of interactive eBook features may result in more appropriate and different 
sets of results. As with any other study, this study also has limitations. And one of the limitation here is, since some 
students had not used an interactive eBooks before, they had to make guesses on several answers. The researcher 
argue that it is that general lack of understanding about interactive eBook features that makes this study worthwhile. 

The recommendation here is to conduct similar studies which will have focus upon groups divided 
according to the major field of graduation, prior knowledge, owned device and compare the preferences between 
trained and untrained learners of interactive eBooks features. Furthermore additional studies can be carried out on 
features that grab the attention in terms of look and feel, interactivity and responsiveness. 

References 

Al-Fahad, F. (2009): Students' attitudes and perceptions towards the effectiveness of mobile learning in King Saud 
University, Saudi Arabia. TOJET: The Turkish Online Journal of Educational Technology, 8 (2), 111-119 
Berg, S. A., Hoffmann, K., & Dawson, D. (2010). Not on the same page: Undergraduates ' information retrieval in 
electronic and print books. The Journal of Academic Librarianship, 36(6), 518-525. 

Bierman, J., Ortega, L., & Rupp-Serrano, K. (2010). E-book usage in pure and applied sciences. Science & 
Technology Libraries, 29(1-2), 69-91. 

Binas, M., Stancel, P., Novak, M. and Michalko, M., (2012). Interactive eBook as a Supporting Tool for Education 
Process. 10th IEEE International Conference on Emerging eLearning Technologies and Applications, 
November 8-9. 

Coyle, K. (2008). E-Reading. The Journal of Academic Librarianship, 34(2), 160-162. 

Global Web Index (2013): Stream Report: Social (Q3 2013) Retrieved from 

https://www.globalwebindex.net/products/report/stream-social-q3-2013 
Grimshaw, S., Dungworth, N., McKnight, C., & Morris, A. (2007). Electronic books: Children ’s reading and 
comprehension. British Journal of Educational Technology, 38(4), 583-599. 

Haiguang, F., Chenzhu, G., Pan, L., BaoCong, .J, (2012). The Research on E-book-oriented Mobile Learning System 
Environment Application and Its Tendency. The 7th International Conference on Computer Science & 
Education (ICCSE 2012) July 1417, 2012. 

Huang, Y. M., Liang, T. H., Su, Y. N., & Chen, N. S. (2012b). Empowering personalized learning with an 

interactive e-book learning system for elementary school students. Educational Technology Research and 
Development, 60(4), 703-722. 

Korat, O. (2010). Reading electronic books as a support for vocabulary, story comprehension and word reading in 
kindergarten and first grade. Computers & Education, 55(1), 24-31. 

Korat, O., & Shamir, A. (2007). Electronic books versus adult readers: Effects on children ’s emergent literacy as a 
function of social class. Journal of Computer Assisted Learning, 23(3), 248-259. 

Korat, O., & Shamir, A. (2008). The educational electronic book as a tool for supporting children 's emergent literacy 
in low versus middle SES groups. Computers & Education, 50(1), 1 10-124. 

KSU library (2014): King Saud University online library, Retrieved from http://library.ksu.edu.sa 
Lam, P., Lam, S. L., Lam, J., & McNaught, C. (2009). Usability and usefulness of eBooks on PPCs: How students' 
opinions vary over time. Australasian Journal of Educational Technology, 25(1), 30-44. 

NCEL (2014): The National Center for e-Learning and Distance Learning, Retrieved from 

http://elc.edu.sa/portal/?q=en/aboutus 

Pattuelli, M. C., & Rabina, D. (2010). Forms, effects, function: LIS students’ attitudes towards portable e-book 
readers. Aslib Proceedings, 62(3), 228-244. 

Segal-Drori, O., Korat, O., Shamir, A., & Klein, P. S. (2010). Reading electronic and printed books with and without 
adult instruction: Effects on emergent reading. Reading and Writing, 23(8), 913-930 
Shamir, A., Korat, O., & Barbi, N. (2008). The effects of CD-ROM storybook reading on low SES kindergarteners ’ 
emergent literacy as a function of learning context. Computers & Education, 51(1), 354-367. 

Woody, W. D., Daniel, D. B., & Baker, C. A. (2010). E-books or textbooks: Students prefer textbooks. Computers & 
Education, 55(3), 945-948. 

Zajac, M., (2009): Using learning styles to personalize online learning, Campus-Wide Information Systems, 26 (3), 
256 - 265 


13 


Recognition of Prior Learning Occurring in Online Informal and Non-Formal 
Learning Environments: The Case of Higher Education in Turkey 


Mesut Aydemir 

mesuta@ogu.edu.tr 

Eski§ehir Osmangazi University, Eski§ehir, Turkey 

Introduction 

70-90% of individuals' daily formal education learning takes place in informal learning environments like home, job 
or communities (Latchem, 2013). As Recognition of Prior Learning (RPL) provides new opportunities in different 
contexts by highlighting the importance of knowledge and learning acquired from online informal and non- formal 
learning environments, the supremacy of formal education has started to be challenged lately. Policy and research in 
RPL is often premised on the notion that learning from experience can be made equivalent to academic learning 
(Shalem and Steinberg, 2002). Although varying in higher education field throughout the world in terms of its online 
implications, as in the rise of Massive Open Online Courses (MOOCs) and Open Educational Resources (OER) 
movement, RPL has been an important part of higher education policy for a long time with a rising interest. Besides, 
regarding the fact that social networks support various learning types as well as innovative educational practices, 
their potentials in informal and non-formal learning have come into focus (Lucas and Moreira, 2009). When we look 
at the scope of the PLR practices in the world, it can be seen that there are mainly two types of RPL: Credit Transfer 
(CT) and Prior Learning Assessment and Recognition (PLAR). Additionally, assessment, accreditation and 
recognition of prior learning suggests the process of confirming adults’ prior learning in a variety of informal or 
non-formal contexts, with emphasis on learning taking place outside formal educational institutions (Stenlund, 

2010). This process of ‘making learning visible’ (Bjomavold, 2000) has been given different names in various 
countries. As an example, in the UK, the accepted term is accreditation of prior (experiential) learning, in the USA 
and Canada it is prior learning assessment (although Canada rarely uses accreditation of prior learning) and in 
Australia, New Zealand and South Africa the term recognition of prior learning is used. Even if the concept varies, 
the general idea behind these concepts can be summarized as recognizing and acknowledging individuals’ skill and 
experience regardless of how and where it is obtained. Recently, many worldwide organizations, commissions and 
enterprises have made emphasis on the RPL. Although Turkey is a member of one these enterprises, namely 
European Higher Education Area, and the recognition of prior learning is the main means to promote lifelong 
learning in Turkey, there has been little or no improvements with this regard mainly due to the legislation’s not 
allowing recognition of informal and non-formal learning officially (Omer, 2009; Lafont and Pariat, 2012). As an 
example, in foreign language education discipline prior learning is recognized and assessed while in many other 
disciplines it is not. With this in mind, the purpose of this study is to discuss the RPL types and degrees to which 
RPL is accomplished. Moreover, the study reviews the related key implementation issues deriving from the concerns 
in higher education in Turkey and proposes possible solutions within the scope of recognition of prior learning 
taking place in informal and non-formal online learning environments. 

Literature Review 


What is Prior Learning? 

UNESCO defines prior learning as “the formal acknowledgement of skills, knowledge, and competencies that are 
gained through work experience, informal training, and life experience” (Vlasceanu, Griinberg, & Parlea, 2004, p. 
55). According to Gambescia & Dagavarian (2007), there are five categories of prior learning: 

1 . Awarding transfer or awarding of credit from a regionally accredited institution - This is the most 
common type of prior learning credit award. Basically it is a transcript review of learning from another 
institution. Institutions accept in “good faith” with the understanding that courses may not be absolutely 
equivalent to their own but they are of equivalent value. 

2. Awarding transfer or awarding of credit for advanced standing from an accredited (but not regionally 
accredited) post-secondary institution for formal documented learning at the college level. This is basically 
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the same as the first, but often involves a non-regionally accredited institution such as a seminary, or a 
technical or professional school, etc. 

3. Awarding of credits through course challenge exams sponsored by department. This enables students to 
take an exam to advance more quickly to another level of learning. 

4. Awarding of credits by taking nationally recognized standardized exams that assess content knowledge 
of college-level courses such as College-Level Examination Program (CLEP), DANTES Subject 
Standardized Tests, Advanced Placement Exams (AP), etc. This is a very common approach in adult 
completion undergraduate programs. Students take these exams at scheduled, proctored sites and the scores 
are then used by the institution to indicate college-level course knowledge achievement. 

5. Awarding of credits for formal and well-documented training programs conducted by non-collegiate 
sponsors such as tests, certification examinations or certificates that are recognized by, for example, the 
American Council of Education, for which the institution may choose to award credit or advanced 
standing. That decision is entirely based on the institution and is not required by any state or federal 
regulations. 

6. Awarding of credits from assessment of prior learning gained from work/life experience. This often 
takes the form of a portfolio or experiential reflective essay or a set of documentations in which the student 
assesses his/her learning and demonstrates that it meets substantive college-level course learning. 

Recognition of Prior Learning (RPL) 

There has been an increasing understanding that learning from experience, gained in a variety of informal or non- 
formal environments including work and society, should be more largely recognized and valued, especially in 
educational institutions (Castle & Atwood: 61). Additionally, university-level learning can take place outside the 
formal educational institutions and this learning can be expressed and evidenced and should be formally recognized 
(Alexandre, 2014). 

Why is it important? 

As Stenlund (2010) states a lot of effort has been put into the development of procedures to assess prior learning, 
regardless of whether the learning has taken place in formal, informal or non- formal settings. However, considerably 
less consideration has been given to research in this area. Additionally, the formation of systems for the recognition 
and validation of all forms of learning, regardless of their environment, has become one of the significant topics in 
all parts of education and training. The importance of learning that takes place outside of the classroom can be 
mainly categorized for learners and educational institutions. For learners it has the following benefits (Wynne, 

2014): 

• Facilitates access for 'non-traditional' students - people who may not have the opportunity to do further 
study can obtain higher qualifications 

• Acknowledges value of learning outside a formal setting, e.g. values and recognizes learning in the 
workplace 

• Validates the worth of learning students have achieved by themselves 

• Enables progression to other programs of study 

• Eliminates unnecessary repetition and duplication of material already familiar to the student. Public (and 
private) money is better used because people who already have skills and knowledge are not re-trained. 

• Shortens the time necessary to earn a qualification - this motivates students who might otherwise be 
discouraged by the length of time required to complete a college level course or a particular program of 
study 

• Enhances students' pride and self-esteem for what they have accomplished as learners 

• Enhances students' perception and understanding of learning as a lifelong process 


15 



For higher education institutions it, 

• increases institutional efficiency by eliminating the need for unnecessary training 

• increases student recruitment and retention 

• allows for more appropriate learner placement in programs 

• increases accessibility to a broad range of learners 

• provides an important service for business, community and industry 
Retrieved from http://www.rrc. ca/index.php?pid^658 on 12.05.2014) 

Learning Types 

In order to have a better understanding and sight into RPL, it would be beneficial to have a look at different learning 
types. 

Online Learning 

Both in definition and practice, open and distance learning (ODL) has become identical with modem models of 
program delivery that offer more generous open and flexible learning opportunities to wider and more different 
individuals than traditional classrooms provided (Conrad, 2008). Thanks to the rapidly developing Internet and 
Communication Technologies (ICT) and initiatives like Open Educational Resources (OER) and Massive Open 
Online Courses (MOOCs), access to any information has become extremely easier. While this is the case, it is 
common for learners to fully make use of the flexibility feature of online learning which enables individuals to 
acquire knowledge. As a result, it becomes inevitable for learners to demand recognition of their online acquired 
knowledge, which can be under the term prior learning. 

Life-Long Learning 

According to the European Commission, lifelong learning is defined as "all learning activity undertaken throughout 
life, with the aim of improving knowledge, skills and competence, within a personal, civic, social and/or 
employment-related perspective." From this definition, there is an emphasis on the continuation of the learning 
process through out one’s life. With this in mind, it is clear that learning is a never-ending process and thus there is 
certainly an organic link between lifelong learning and recognition of it. 

Formal Learning 

According to OECD definition, formal education is defined as “education provided in the system of schools, 
colleges, universities and other formal educational institutions and that normally constitutes a continuous ladder of 
full-time education for children and young people, generally beginning at age 5 to 7 and continuing up to 20 or 25 
years old or above.” This learning type is probably the least problematic type in recognition of prior learning simply 
because it is fulfilled within a formal context (e.g. schools or universities) and offers an accredited certificate or 
diploma, which makes the recognition process easier compared to other learning types. 

Informal Learning 

Informal and incidental learning is at the heart of adult education because of its learner-centered focus and the 
lessons that can be learned from life experience (Marsick and Watkins, 2001). Informal learning is usually 
intentional but not highly structured. Self-directed learning, networking, coaching and mentoring, can be given as 
examples. Informal learning, which should be regarded as the ancestor of formal learning, shouldn’t be considered 
as a lower form of learning. Furthermore, it needs to be seen as fundamental, necessary and valuable type of 
learning occurring in everyday life contexts. (Coffield 2000) 

Incidental Learning 

Incidental learning almost always occurs although people are not always conscious of it (Marsick and Watkins, 
1990). Although there exists a slight difference between informal and incidental learning, according to a research 
conducted by Callahan (1999) informal and incidental learning show similarity in practice in many cultures and 
contexts such as the private and public sectors, hospitals and health care, colleges and universities, schools, 
professional associations, museums, religions, families, and communities. 
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Non-Formal Learning 

According to the OECD definition, non-formal education is defined as any organized and continued educational 
activities that do not match exactly with the definition of formal education. Non-formal education may therefore 
occur both within and outside educational institutions, and serve to learners from all ages. Depending on different 
countries, it may vary in its coverage. Non-formal education programs do not necessarily follow the hierarchy 
system, and may have a differing time length. 

Since recognition of non-formal and informal learning touches on many different issues within the education 
system, although the term ‘recognition’ is a single word, its meaning is not. The main goals followed through the 
process of recognition of non-formal and informal learning differ according to the individual and country, from 
exemption of all or a part of the entry requirements for a formal learning program, to the awarding of a full 
qualification (Werquin, 2008). 

Personalized Learning 

According to the US Department of Education, personalized learning is defined as adjusting the pace 
(individualization), adjusting the approach (differentiation), and linking to the learner's interests and experiences. 
Personalization is more than just individualization or differentiation because it allows the learner a degree of choice 
about what is learned, when it is learned and how it is learned, which is often expressed as learning 'any time, any 
where or any place'. This should not be regarded as unlimited freedom, because learners will still have targets to be 
accomplished. However, it has the advantage of providing learners the opportunity to learn in ways that suit their 
individual learning styles. Hargreaves and Shirley (2009), however, warn about potential negative aspects of some 
dimensions of personalized learning. As an example they state that although it is ok for learners to be able to access 
information instantly online, this process should not be regarded as a deep, critical and meaningful learning. 
Personalized learning can take place in collaborative activities, for example a group of learners working together to 
study a particular topic, which can be stated as the one of the targets of connectivist Mooes. ICT can be a powerful 
tool for personalized learning as it allows learners to do research and enables them access to information, and 
provides a method for communication, discussion, and recording learning achievements. The general discussion is 
that there has been much focus on technological tools rather than the spirit of personalized learning. However, all of 
the different parties of this discussion on personalized learning come to a consensus that formal learning 
environments should become more flexible and adaptive in responding to the diverse needs and interests of students. 

Current situation in some parts of the world 

Recognition of Prior Learning can be analyzed mainly under two institutions in the US: Community Colleges and 
Universities. Two-year community colleges typically function for adult students who are generally excluded from 
the educational system. These colleges are also places where many community workers find a place for their higher 
education. For these people, prior learning is valuable for access and credit. Community colleges believe that adults 
are life-long learners who actively build knowledge from experience; and therefore, assessment of prior learning for 
credit is an integral part of their educational process. Many community colleges have special courses that help guide 
students through the process of identifying, documenting and assessing their prior learning. Universities in the US 
typically have different implications in the process of recognition of prior learning. The main idea is that, they 
generally get applications from the learners and assess the applicants with some extra supportive courses, trainings 
or workshops prior to accrediting or accepting (Alexandre, 2014). 

According to the OECD Report (2010) recognition of prior learning plays an important role in a number of countries 
by offering validation of capabilities to simplify entry to further formal learning process. This generally includes 
exemption from certain coursework or parts of a formal learning environment. It also allows learners complete 
formal education more quickly, efficiently and cheaply by not having to enroll in previously taken courses 
(Werquin, 2008). In the same report, it is also stated that, although the 22 countries, which contributed to the study, 
are very well aware of the importance of RPL, they generally differ in their applications and policies regarding it. 
Moreover, there has been little or no research on this specific area and thus both countries and educational 
institutions are seeking for frameworks on RPL. The following figure would provide a graphical view of the present 
situation of some of the countries. 
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Prior learning 

□ can be used towards fulfilment of a 
HE study programme 
H cannot be used towards fulfilment 
® of a HE study programme 

Data not available 


Figure 1. Recognition of prior non-formal and informal learning for progression in higher education studies, 
2010/11, European Commission Eurydice Overview 

Discussion and Conclusion 

When looking into the related literature on RPL, related issues can be categorized under accreditation, research and 
policy. First of all, educational institutions are generally reluctant in accrediting prior learning. Although today's 
learners, especially adults, learn in various ways throughout their lives, their social skills develop over years and 
additionally, as needs arise their technical skills develop in informal and non-formal environments such as work, 
home and online networks. However, they are generally facing difficulties in accrediting their prior learning. A 
collaborative approach among different stakeholders and external accreditation can solve this issue. 

Secondly, there is little experiential research on the effect of prior learning on learners and/or on higher education 
institutions (Alexandre, 2014). This might be one of the reasons why present stakeholders of education, namely 
learners, educational institutions, faculty members, employers and policy makers seem to be unwilling on 
welcoming RPL. A more detailed review that may result in richer documentation will certainly help to overcome 
this reluctance. Thorough research and studies will certainly ease the acceptance, application and embracement of 
RPL in existing implications. 

Thirdly, no matter how much importance learners, educational institutions, faculty members or even employers 
attach on RPL, unless it takes its place in policy in the current systems of countries, it will not go beyond being one 
of the debates in the field of education. Hopefully there is a tendency that countries are becoming more willing to 
recognize non-formal and informal learning outcomes within the existing educational system (Werquin, 2008). 
Defining and improving recognition procedures and processes through collaboration among the stakeholders will 
result in effective policies with this regard. 
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Abstract 

This paper describes a qualitative look into the content surrounding the open education Twitter hashtag 
(#openeducation) using content analysis methodology. A convenience sample of 903 tweets using the 
#openeducation hashtag was obtained and their content was inductively coded using open coding. The content of 
these tweets were analyzed for primary topic and emerging themes were organized into categories. These data were 
tallied along with the number of contributions made by each user of the #openeducation hashtag. The resulting 
themes and categories were used to guide the development of a questionnaire which was subsequently provided to 
the most active users of the #openeducation hashtag. A brief literature review was then conducted to ground the 
emergent themes and categories into the literature surrounding open education. 

Abstract 

This paper describes a qualitative look into the content surrounding the open education Twitter hashtag 
(#openeducation) using content analysis methodology. A convenience sample of 903 tweets using the 
#openeducation hashtag was obtained and their content was inductively coded using open coding. The content of 
these tweets were analyzed for primary topic and emerging themes were organized into categories. These data were 
tallied along with the number of contributions made by each user of the #openeducation hashtag. The resulting 
themes and categories were used to guide the development of a questionnaire which was subsequently provided to 
the most active users of the #openeducation hashtag. A brief literature review was then conducted to ground the 
emergent themes and categories into the literature surrounding open education. 

Introduction 

The rise in digital technologies such as the Internet and hypertext has had an impact on nearly every industry, 
field, and topic in the world. Its effects can be seen in the norms, communication patterns, and social cultures that 
surround these industries, fields, and topics. Social Media is a connective technology that enables communication 
between individuals and groups who may not have otherwise ever connected (Ravenscroft, 2011). Thanks to social 
technologies, learners in various settings all over the world often interact with people, perspectives, and content that 
they otherwise may never have encountered. The social fabric that develops as a result of these unique connections 
and encounters are a major part of the digital culture surrounding social media. 

Another culture driven by connection and empowered in many ways by social media is the open education 
movement (Baker III, 2014). People form this culture through connecting with others on a wide variety of ideas, 
influences, founding philosophies, beliefs, tools, experiences, drives, and perspectives. The open education 
movement is a grand experiment able to test innovative ideas and learning designs that are often either not possible 
or feasible to experiment with inside of the constraints of traditional education structures (Baker III, 2014). 
Connective digital technologies often support these experiments through enabling more accessible and transparent 
participatory open classrooms that often exhibit reduced barriers and built-in systems for obtaining feedback from a 
variety of perspectives (Baker III & Surry, 2013). 

Twitter is an example of a connective digital technology that is commonly used in open education courses. A 
Twitter hashtag is a feature of the Twitter platform where people who do not necessarily know each other but all 
have Twitter accounts or access to a twitter feed can come together around a topic of interest by searching for the 
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hashtag term. A hashtag term is plaeed into a tweet (i.e., a post 140 eharaeters or less to the Twitter platform) and is 
made up of a pound sign (#) and a word of interest following the pound sign (e.g., #openedueation is used for people 
interested in open edueation, #edteeh is used for people who are interested in edueation teehnology, and #oer is used 
for people interested in open edueational resourees). Hashtags aehieve eommon aeeeptanee in groups based on who 
and how many people use them and are often set up to allow people to eonneet at eonferenees, elasses, 
presentations, and other events (Chang, 2010). The eommon hashtags for a given topie ean often be found using an 
Internet seareh (i.e., Google, Bing, Yahoo, ete.). Programs like Tweetdeek, Hootesuite, or Janneter are often used for 
their ability to set up a seareh eolumn that will automatieally seareh and update tweets sent to a given hashtag or 
seareh term. The #openedueation Twitter hashtag is a primary hub of eonneetion for those interested in the open 
edueation movement. This study deseribes an attempt to diseover the answers to the following researeh questions: 

• How is the #openedueation hashtag is used? 

• What are the most popular topies using the #openedueation hashtag? 

• Are the most aetive hashtag eontributors also aetive in open edueation? 

Methods 

This study was designed to diseover the eontent, prevalenee of topies, and most aetive eontributors related to the 
#openedueation hashtag. The researeh design deeisions were guided by the design methodologies in Creswell (2013) 
and Johnson & Christensen, (2008). The website Topsy.eom arehives tweets for a variety of hashtags and sells 
aeeess to the eomplete reeords for a given hashtag to interested parties. The quote to obtain the entire data set was 
well out of the priee range for this study, however the site provides aeeess to a subset of tweets surrounding a given 
hashtag using a seareh feature. The Topsy database began arehiving tweets sometime around late 2009 to early 
2010, and they have reeords for the #openedueation hashtag dating baek all the way to 2010. 

A eonvenienee sample of 903 tweets using the #openedueation hashtag were obtained from the Topsy.eom 
website’s hashtag seareh. The sample was obtained using the advaneed seareh feature on the Topsy seareh software 
on their website (www.topsy.eom). The seareh term “#openedueation” was used in the “All these words:” seetion, 
with the settings “Seareh a speeifie type:” set to “Tweets”, the “Seareh a speeifie language:” set to “English” and the 
“Sort results by:” option set to “Date,” as displayed in Figure 1. These tweets were all posted to Twitter between 
2009 and 2012, were in the English language, and provided the eoding pool of eommon themes and the most aetive 
users in our sample. 


TOPSY 


Advanced Search 


All these words: 

Any of these words: 

None of these words; 

Search within a site or domain; 
Search within a Twitter user: 
Search a specific type: 

Search a specific language: 
Sort results by: 

Oldest date and time: 

Most recent date and time; 


#optneducation 


Tweets : Show only trending results 

I Engli sh 

Date t 

' J I 12AM : i 

1 I f 12AM : 1 rj, 

I Search I 


Learn more about search operators available on Topsy. dick here 


Figure 1. The advanced search settings used at Topsy.eom 

This is a mixed methods study with a qualitative emphasis derived from descriptive quantitative elements 
(quan^QUAL). All tweets from the sample were analyzed to obtain the emergent themes, categories, and most 
active users present in the #openeducation hashtag sample data. Throughout each stage of the study, every attempt 
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was made to interact with the data from both the emic and etic perspectives. There is no way to know whether these 
sample data are randomly derived from the #openeducation hashtag tweet population because the Topsy.com search 
feature is outside of the researchers control; therefore, statistical generalization, which is neither of primary concern 
or a purpose of this study, is limited. 

During data analysis, the tweets from the sample were analyzed by content (e.g., sharing a link to an article or 
lecture, mentioning a conference experience or competition, stating views on a subject, etc.) and inductively coded 
into themes (e.g., articles, conferences, opinions, etc.) by a single coder using an open coding format. The coding 
sessions were performed in large chunks with relatively short periods (hours or days) between the coding sessions in 
order to minimize intra-coder reliability issues. The themes were enumerated through being tallied as they were 
coded and were later collected into categories (e.g., information sharing, research, open educational content, etc.) 
based on their relatedness. These data provided an idea of the prevalence of each theme and category in the sample. 
The contributing author usernames for each tweet were also listed and their contributions were enumerated and used 
to determine the most active users of the hashtag. The most active #openeducation hashtag users were identified 
from the tally data and contacted via Twitter to participate in a survey. The survey inquired about their Twitter 
usage, blogging participation, employment field, their interest and connection with Massively Open Online Courses 
(MOOCs) and other various topics in open education and related topics. Three users agreed to participate in the 
questionnaire. 


Emerging Categories and Themes 

The data analysis brought about thirty-two themes that fit well into eight major categories based on their 
relatedness. In this paper, “theme” refers to the overarching concept name or code (e.g., link to article, blog posts, 
discussion/lecture) given to a common topic (e.g., links to articles, blog post mentions, linking to a lecture, etc.) 
found in the content analysis of the sample tweets. “Category” refers to the overarching concept name or code (e.g., 
information sharing, open educational designs, change & awareness, etc.) given to a collection of themes based on 
their relatedness. These categories are representative of major areas in open education and have a variety of presence 
in the data. 
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Category 

% 

Item 

Count 

% 

Cum % 

Information 

Sharing 

21.04% 

Links to non-research Articles 

148 

16.39% 

16.39% 

Blog Posts 

29 

3.21% 

19.60% 

Opinion 

7 

0.78% 

20.38% 

Ted 

6 

0.66% 

21.04% 

Connection 

10.41% 

Collaboration Request 

17 

1.88% 

22.92% 

Shared Resource 

77 

8.53% 

31.45% 

Research 

9.08% 

Research 

21 

2.33% 

33.78% 

Open Access 

32 

3.54% 

37.32% 

Research Articles 

8 

0.89% 

38.21% 

Conference 

21 

2.33% 

40.53% 

Open 

Educational 

Designs 

24.47% 

Discussion/Lecture 

58 

6.42% 

46.95% 

E-leaming 

9 

1.00% 

47.95% 

Open Courses 

132 

14.62% 

62.57% 

Learning Management Systems 

13 

1.44% 

64.01% 

Open Scholarship 

5 

0.55% 

64.56% 

Accreditation 

4 

0.44% 

65.01% 

Open 

Educational 

Content 

19.60% 

Open Courseware 

25 

2.77% 

67.77% 

Open Educational Practices 

4 

0.44% 

68.22% 

Open Educational Resources 

100 

11.07% 

79.29% 

Intellectual Property Licensing 

9 

1.00% 

80.29% 

Open Text Books 

39 

4.32% 

84.61% 

Change & 
Awareness 

9.19% 

Education Reform 

19 

2.10% 

86.71% 

Policy 

17 

1.88% 

88.59% 

Competition 

18 

1.99% 

90.59% 

Open Production 

1 

0.11% 

90.70% 

Open Education Week 

15 

1.66% 

92.36% 

Open Data 

7 

0.78% 

93.13% 

Open Science 

6 

0.66% 

93.80% 

Open 

Technology 

4.32% 

Virtual Worlds 

3 

0.33% 

94.13% 

Technology 

4 

0.44% 

94.57% 

Open Source Software 

32 


98.12% 

Business 

Promotion 

1.88% 

Promotion 

17 

1.88% 

100% 


100% Total 903 100% 


Table 1. Categories and themes that emerged from the data, and their pereent eontribution to the total. 

As seen in Table 1, the eategories, with their pereent of contribution to the total makeup, are Open Educational 
Designs (OEDs) (24.47%), information sharing (21.04%), Open Educational Content (19.60%), connections 
(10.41%), change & awareness (9.19%), research (9.08%), open technology (4.32%), and business promotion 
(1.88%). The most prominent themes were people sharing non-research articles (16.39%), open courses (14.62%), 
open educational resources (1 1.07%), people sharing resources (8.53%), people sharing their discussions & lectures 
(6.42%), open textbooks (4.32%), open source software (3.54%), open access research (3.54%), open courseware 
(2.77%), conferences (2.33%), and research (2.33%). Among the least discussed topics were open production 
(0. 1 1%), virtual worlds (0.33%), pure technology (0.44%), and open educational practices (0.44%), which is noted 
as the next evolution of open educational resources (OERs) (Open Educational Quality Initiative, 2011). OEDs is an 
umbrella term for courses designed to be open to some degree through incorporating transparency and freedom 
(Baker III, 2014; Baker & Surry, 2013), where OER are content items that can be reused, remixed, redistributed, and 
revised due to open licensing permissions (Wiley, 2010). 
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While the makeup of the emerging categories is interesting, the makeup of activity in each category is even more 
interesting. The most active category was OEDs, with nearly 25% of all tweets in the sample discussing something 
related to open educational course design, where the most active theme (by 1.77%) was people sharing non-research 
articles (16.39%). Examining the percentage breakdowns of these themes provides an interesting insight into the 
open education culture. It becomes evident that sharing informational resources is important, and that educational 
structures are a prominent topic. Considering the dominant categories and topics can provide important perspectives 
for speculating on the most important values held in the cultural makeup surrounding the #openeducation hashtag. 
According to these data, concepts related to Open Educational Designs (24.47%), Information Sharing (21.04%), 
and Open Educational Content (19.60%) seem to be the dominant concerns in the culture. 

The emerging themes and categories are consistent with ideas found in the literature on open education. Open 
content (Butcher, 2011; Carnegie Foundation for the Advancement of Teaching, 2008; Morgan & Carey, 2009; 
Wiley, 2012), open courses and Open Educational Designs (Baker III & Surry, 2013; Dougiamas & Taylor, 2003; 
Hannafin, Land, & Oliver, 1999; Kikkas, Laanpere, & Poldoja, 2007; Mentor, 2007; Morgan & Carey, 2009; 
Rodriguez, 2012; Times et al., 2012), assessment strategies in open designs (Gray, Thompson, & Sheard, 2010; 
United Nations Educational Scientific and Cultural Organization, 2002), open access research (Carnegie Foundation 
for the Advancement of Teaching, 2008; Christensen, 2005; Hedlund, 2011; Krikorian, 2010; Lewis, 2012; 
Mukheqee, 2009), Open Educational Practices (Open Educational Quality Initiative, 2011; Open eLearning Content 
Observatory Services & Geser, 2007; Piedra & Chicaiza, 2009),and Open Educational Resources (Caswell, Henson, 
Jensen, & Wiley, 2008; Ehlers, 2011; Porter, 2012) are all prime and relevant topics in the open education literature. 
Connection, sharing, and social technologies are also prominent in the open education literature relevant to 
pedagogy, scholarship, and learning through connected/networked teaching and learning (Bell, 2010; Couros, 2009; 
Drexler, 2010; Mattar, 2010; Veletsianos & Kimmons, 2012). Open education has an interesting history (Baker III, 
2014; Mcnamara, 2012; Meiszner, 2011; Shelley & Sherman, 2009) and openness is poised to impact both higher 
education (Digital Connections Council, 2009; Wiley, 2006, 2010) and the larger society (Baker III, 2012; Lessig, 
2004, 2008) in profound ways. 

Overall, the themes discovered in the content analysis appear to corroborate well with the literature. The analysis 
reveals that common themes for the #openeducation hashtag content are non-research articles related to open 
education (16.39%), content related to open courses (14.62%), and content related to open educational resources 
(1 1.07), where the most active categories have to do with Open Education Designs (24.47%) and information 
sharing (21.04%). These are also common themes and categories in the literature. It appears from these data that the 
#openeducation hashtag community actively deals with similar content as the literature surrounding the open 
education movement. 

Questionnaire Responses 

The theme and category data that emerged from the hashtag analysis were used as a guide for developing the 
questionnaire used in the second phase of the study. The questionnaire addressed the major categories and concepts 
found in the data. It asked about the participants’ social media presence and habits, participation in MOOCs, use and 
development of open content, field of employment, level of education attained, and some general demographics 
questions. The nine most active users of the #openeducation hashtag identified in the data were contacted via 
Twitter to participate; three of those contacted responded to the questionnaire requests (33.33%). 

The questionnaire respondents reported starting their Twitter accounts in 2008, 2009, and 2011. Twitter was 
founded in 2006 based on open source software, and currently has approximately 200 million active users 
(@Twitter, 2012), so the respondents do not immediately fall into early or late adopter categories. There were two 
male respondents, and one female, and the respondents indicated age ranges of 25-29, and 45-49 (one chose not to 
respond). They also reported being from Europe, the UK, and Australia. 

As seen in Table 2, the respondents are active in different areas of open education. All respondents were active 
on Twitter, two were active bloggers, and two had actively used and developed Open Educational Resources. One 
had previously participated in Massively Open Online Courses (MOOCs), including a Coursera course, a Udacity 
course, the EdStartup 101 course, and the MOOCMOOC course. These MOOCs vary in style, underpinning 
philosophy, and design, and represent a variety of different approaches to open education (Baker III & Surry, 2013). 
This person was also the only respondent who worked in higher education. It is interesting to note that, as seen in 
Table 3, the questionnaire respondents showed limited interest in the largest analysis category (i.e.. Open 
Educational Designs), but all respondents expressed interest in the smaller category of Open Educational Content. 
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Activity 

Respondent 1 

Respondent 2 

Respondent 3 

Year Began Twitter 

2008 

2009 

2011 

Twitter Role 

Individual: 
Personal & 
Professional 

Individual: Personal & 

For Profit 

Professional 

Organization 

# Tweets/Day 

10+ 

20+ 

5+ 

Tweets related to Open 
Education 
(#openeducation) 

10% 

10% 

40% (10% use 
#openeducation) 

Other Flashtags 

#edtech 

#oer/ “several 
thousand” others 

#opened 

% Retweets 

10% 

10% 

10% 

Active Blogger? 

Yes (Educational 
Technology) 

Yes (Open Education) 

No 

MOOC Participation? 

Yes 

No 

No 

Develop OER? 

Yes 

No 

Yes 

Use OER? 

Yes 

No 

Yes 

Fligher Education? 

Yes 

No 

No 

Flighest Degree 

Master’s Degree 

Doctorate 

Master’s Degree 


Table 2. Summary table of questionnaire respondent’s activity related to open education topics 


Category 

Item 

% 

OED 

Topic Focused Models 

33.30% 

Alternate Education Models 

33.30% 

Flipped Classroom 

33.30% 

Research 

Open Access Research 

100.00% 

Open Content 

Open Content 

100.00% 

Open Educational Resources 

100.00% 

Open Educational Practices 

33.30% 

Open Textbooks 

100.00% 

Intellectual Property Licensing 

100.00% 

Open Courseware MIT 

66.60% 

Change and Awareness 

Open Policy 

33.30% 

Open Science 

66.60% 

Open Data 

100.00% 

Technology 

Emerging Tech/Social Media 

33.30% 

OSS 

66.60% 


Table 3. Summary table for questionnaire respondent’s interest areas in open education topics 


Discussion 

Together, the content analysis and the responses to the questionnaire provide sufficient data to start constructing 
speculative responses to the research questions this study sets out to answer. From these data, it is possible to see a 
few common characteristics emerge that speak to how the #openeducation Twitter hashtag is used by the culture 
surrounding it. Based on the literature, the larger open education community values sharing, collaboration, 
connection, and innovation, and these themes follow through to the #openeducation hashtag as evident in the content 
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analysis and questionnaire responses. The collaborative element is enhanced by the ability to connect through 
technology and seems to occur both formally and informally. Sharing using the #openeducation hashtag is a 
common practice as evidenced by the number of shared resources and article shares in the sample data. The 
questionnaire respondents were also bloggers, tweeters, and contributors to open education, which shows their 
interest in sharing ideas. From this, it is evident that the #openeducation hashtag is used to enable and support this 
collaboration, connection, and sharing. 

From these data it is also apparent that Open Education Design and Open Educational Content are important 
topics on the #openeducation hashtag, and the practice of information sharing is also a large part of the community. 
Linking to non-research articles and content regarding open courses and OER were prime topics. People also used 
the #openeducation hashtag to share access to discussions and lectures. The #openeducation hashtag is often used to 
connect with others, share perspectives, and discuss these and other topics of interest between parties who are part of 
a similar culture of interest but who may not otherwise know each other. 

The most active contributors to the #openeducation hashtag contribute to open education in a variety of ways. 
They are active in blogging, using and developing OER, advocating for and disseminating information about 
openness, and take part in the culture in a variety of ways. Based on these data, the prevalence of a person’s use of 
the #openeducation hashtag is also indicative of their larger efforts in open education. An interesting opportunity for 
future research is to follow up on this study and intentionally analyze the types of tweets made by the most active 
tweeters as they relate to the specific activities related to open education in the larger community. Overall, the 
#openeducation hashtag culture and the larger open education movement culture appear to have common threads 
woven between them. These threads manifest themselves in various ways (e.g., sharing knowledge, collaborating 
publicly, networking, developing resources for public use, etc,) in each culture and surround a very human element 
centered around connecting with others; even when they are separated by spatial, temporal, cultural, educational, 
and lingual barriers. These connections are enabled by social media and digital environments where people can 
connect, and allow for new ways of collaboration and sharing to emerge. 

Suggestions for Future Research 

This study provides an example of a content analysis and questionnaire model useful for obtaining perspectives 
on cultural characteristics through analyzing social media hashtag products. The study also provides a model for 
identifying the most active members of a social media culture and generating the themes and categories relevant to 
the common behaviors. These themes and categories can then serve as a guide for developing questionnaires that 
target the interests and cultural elements most relevant to the culture under study. Unfortunately, the option choices 
utilized in this study’s questionnaire did not enable an equal comparison between the questionnaire data and the 
content analysis data; therefore, an opportunity for future researchers exists to repeat the study with a more robust 
questionnaire instrument that can make a more direct comparison between the content analysis and the 
questionnaire. Additionally, the cost for obtaining the full data set from Topsy.com far exceeded the resources 
available for this study. Future researchers could develop a grant or look into alternative means of obtaining the 
entire tweet history for the #openeducation hashtag. Perhaps the entire data set will be available in the near future, as 
the Library of Congress is currently developing methods for storing and making searchable the entire history of 
tweets and have approximately 170 billion to date (Library of Congress, 2013). Repeating this study with the entire 
data set more fully sampled may provide more granular results and may confirm the findings in this study. Yet 
another opportunity for future research could be to validate the model used in this study by using it in another study 
either on a different hashtag and culture group or on one that is related to the #openeducation culture in some way 
not utilized in this study. Finally, future researchers may wish to focus on a sub-category or theme from the findings 
in this study and perform a more focused study on a more specific topic, such as Open Educational Designs or Open 
Content, ft would be interesting to start learning about the overlap in cultures between the different categories that 
emerged in the data. 


Conclusion 

Even through distance and non-acquaintance, digital technologies enable ways for people to connect with others 
around content and ideas in innovative new ways. What is more, social media and open education cultures are 
converging to enable connections and collaborations around education and instructional design that would not have 
taken place outside of such a connective social network of technology. This paper describes a study that examined 
the #openeducation hashtag in an attempt to learn more about how it is used, what topics are most popular on the 
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hashtag, and whether the most active hashtag contributors are active in open education. The researcher considered 
the literature on open education and obtained a convenience sample from the #openeducation Twitter hashtag. Three 
of the most active tweeters to the #openeducation hashtag responded to a questionnaire, designed from the results of 
a content analysis, which considered how these participants contributed to open education. The study shows that the 
hashtag is an active platform for connecting with others and sharing ideas, that Open Education Designs and Open 
Educational Content are the primary topic areas discussed on the #openeducation hashtag, and that the most active 
hashtag contributors are active voices in open education in a variety of ways. 

The #openeducation Twitter hashtag provides a hub for connecting strangers with similar interests around 
common ideas, a platform for sharing related contributions in other areas, and a resource for tapping into a flow of 
information related to open education. It seems that, even in an increasingly digital world, people are still finding 
ways to assimilate technology into what it means to be human, and are finding ways to use that technology for 
connecting and building new interactions with others. 
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Introduction 

Online learning continues to grow, however, many online courses utilize technology in a manner that fails 
to actively support learning. Existing lessons are merely integrated into technology, without verifying the impact on 
learning and student achievement (U.S. Department of Education, 2007). Neuroscientists report little benefit to 
students of simply adding technology to conventional educational lessons (Robin, 2008), particularly as learners are 
now accustomed to enriched media as a result of the widespread popularity of interactive games, simulations, and 
entertainment. In addition, online courses have been cited as having a loss of spontaneity that helps to make 
education more effective and enjoyable for the learner (Tiene, 2000). Students are twice as likely to drop out of an 
online course than a traditional course (Willging & Johnson, 2004). Reasons cited include too many low level 
assignments and lack of interest in the material (Willging & Johnson, 2004). Previous studies have found that more 
technology alone does not increase learning outcomes; better course design is needed to help online courses gain 
legitimacy, engage students, and increase learning (Rovai, 2003). 

Technology within online education must tangibly enhance courses. It is important to understand the 
educational benefits of an instructional method and incorporate the method in a meaningful manner. With digital 
storytelling, learners become absorbed within the lesson, and this cognitive arousal leads to a greater focus of 
attention and incorporation of knowledge (Green, Strange, & Brock, 2002). Digital storytelling is viewed as a 
popular method for including students’ voices into curriculum. Students enjoy being able to contribute and 
collaborate, aspects that are reflected in popular culture, through the popularity of blogs, wikis, and YouTube. 

Digital storytelling has the capability of engaging learners and improving learning and performance. Digital 
storytelling involves utilizing imagery, audio, video, and interactive digital features, in combination with narration, 
to involve and inspire learners (Chung, 2007; Robin, 2008). Digital storytelling can enrich online learning in a 
variety of ways, either through the addition of different audio-visual components or by making the lesson more 
immersive and participative. As a result, the information resonates more with the learner. This helps stimulate 
learning and knowledge retention (Bou-Franch, 2012). In addition, digital storytelling reinforces digital literacy as 
well as other 2P‘ century skills, like the ability to use technology, information and images to be able to communicate 
more effectively in today’s world (Bran, 2010; Robin, 2008). 

Theoretical foundations support the effectiveness of digital storytelling in education. Scientists have 
determined incorporating narratives into learning is effective because our brains search neural networks for similar 
items (Westwater & Wolfe, 2000). When our brains recognize information, the information is more likely to be 
retained. New information must be connected to existing information and knowledge in order for the new 
information to establish itself within the learner’s brain. This process is termed homeomorphogenesis (Liston, 1994). 
In addition, because content is grouped together to form a story, in a thematic organizational format, information 
becomes more memorable (McGregor & Holmes, 1999). Learners are able to reconstruct and remember information 
that corroborates with past experiences, enabling learners to add to existing knowledge (Psomos & Kordaki, 2012). 
Constructivism theory is based on this concept. Knowledge is constructed when new information comes in contact 
with existing knowledge that has been developed through experiences. 

Many studies have examined the effects of digital storytelling in developing diverse skills across various 
learning disciplines. Students have created stories to gain skills and participate in the learning process (Kearney, 
2011; Kieler, 2010; Maier & Fisher, 2007). This format has been used to connect students with lessons in history, 
language arts, teacher education, journalism, mathematics, law and science (Bou-Franch, 2012; Iseke, 2011; 
Kearney, 2011; Sadik, 2008; Steslow & Gardener, 2011; van Gils, 2005). In addition, studies demonstrate that 
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digital stories helped struggling readers understand text and meaning better (Dreon, Kerper & Landis, 2011), as well 
as offering the ability to connect “marginalized communities” (Maier & Fisher, 2007, p. 176) with learning materials, 
by overcoming literacy barriers. Furthermore, Maor and Fraser (2005) pointed out the ability of interactive stories to 
help students develop investigative skills, and explore a variety of experiences from multiple perspectives. While 
these studies revealed critical educational effects of digital storytelling, the educational effects of digital storytelling 
are believed to be far-reaching. 

In this research, we critically reviewed empirical studies and conceptual papers on digital storytelling and 
examined the impact of this instructional method on student’s cognition and learning outcomes. We also identified 
challenges and issues regarding effective incorporation of digital storytelling into the curriculum. Instmctional 
design suggestions from the literature were synthesized to provide best practices for future integration of this 
instructional method into online courses. We hope to provide educators and course designers with suggestions to 
encourage incorporation of effective digital storytelling activities, as a way to enrich education, further higher 
learning, and increase the value of online courses. 

Research method 

Initially, a search was conducted for articles involving “storytelling,” “digital storytelling” and “interactive 
storytelling” in the EBSCO and Google Scholar databases. The search was later focused on “digital storytelling” and 
expanded to include the EdITLib: Education and Information Technology Digital Library, ERIC, and Web of 
Science, and articles cited within the articles found. A total of 58 articles were selected, and then later culled down 
to 36, when research became more concentrated on digital storytelling’s educational effects and challenges, and 
suggestions for instructional design. Articles that supplied information for the following result section were 
published between 2005 and 2014, and involved education. Included in this analysis were empirical studies, such as 
case studies, and conceptual papers focusing on digital storytelling environments, models, and theoretical 
framework. These articles addressed the issues of educational effects, challenges, and instructional design 
suggestions. 


Results 


Educational effects 

After the collected research studies were analyzed, we identified the nine most important educational 
effects of digital storytelling. We found that the multimodal aspect of digital storytelling offers increased 
opportunities for learning and can lead to improved knowledge retention. Digital storytelling can help reveal and 
build students’ identity. Students’ communication and collaboration skills are improved through working in a 
community of practice. Producing digital stories also helps students develop reflective and critical thinking skills as 
well as build their digital literacy. Creating their own stories, students become more engaged in learning, and 
produce higher quality work. Each of these educational effects is discussed in this section and supported with 
evidence from literature. 

Multimodal learning and improved knowledge retention. The combination of sound, text, and pictures 
offers students an opportunity to connect images with narration, which allows them to better understand the lesson’s 
meaning and increase knowledge retention. Because digital stories are typically housed on the Internet, students are 
able to replay stories, which aids in knowledge retention. It also affords students a way to create more powerful 
narratives compared to a traditional format. Digital storytelling, according to Robin (2008) is an especially good 
method for combining visual images and written text. Alonso, Molina, and Poro (2013) reviewed 30 multimodal 
digital narratives and discovered the combination of pictures, music, and narration found in digital storytelling 
helped convey “highly complex mental images” (p. 382). Sixteen Spanish students made stories for an English 
studies course in Bou-Franch’s (2012) research, with the premise that meaning is made not only through language 
but the combination of sound, text, and pictures. Choi and Johnson (2005) gave support that combining visual and 
audio gave students “additional and complementary information” (p. 217), which aided comprehension and retention. 
By being involved in actively creating a story, ESL students were able to comprehend and integrate knowledge 
better (Tsou, Wang, & Tzeng, 2006). After a digital storytelling workshop and two weeks of storytelling activities, 
student learners’ post-test scores indicated increased knowledge and improved technical skills (Figg & McCartney, 
2010 ). 

Identity. Students come to terms with how to present themselves to others through digital storytelling, since 
there is an emphasis on personal voice. Stories can be created in which students play historical roles or work through 
past experiences. Bou-Franch (2012) observed students trying new identities as they created stories about various 
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time periods and displayed different personas for the task. Bran (2010) noted students’ ability to develop their own 
individual, unique stories when faced with a general prompt (“Message in a bottle”). In Ranieri and Bmni’s (2013) 
study of fifteen teenagers, students initially drew disembodied figures and their stories were very simple. By 
creating digital stories with mobile phones the students were able to explore and strengthen their identity. The use of 
technology helped 80 students explore and develop their identity within a rural community, as they created digital 
stories to show the relation between location and self (Wake, 2012). 

Communication. Digital storytelling involves communication skills in the form of writing and speaking. As 
a result of the richer experience of digital storytelling, English learning students displayed significant improvement 
in English proficiency, critical thinking and motivation (Yang & Wu, 2012). Although Figg and McCartney (2010) 
found it was initially difficult for at-risk students to begin to write scripts, as these struggling students became 
involved in the task they reported writing the story to be one of the easiest parts of the process, and the students’ 
writing skills increased with the creation of digital stories. Creating digital stories allowed 1 1 ELL students to 
practice their reading and writing, oftentimes reading a script several times to ensure they spoke clearly while 
recording (Hur & Suh, 2012). Using digital storytelling in a public speaking classroom enabled students to improve 
their powers of persuasion and public speaking skills (Leopold, 2010). Employing technology inspired students to 
“write for an authentic audience and recognize their product as something worth sharing with others interested in 
their message” (Wake, 2012, p. 35). Students using Second Life and Windows Movie Maker to write digital stories 
improved their writing self-efficacy (Xu, Park, & Back, 2011). 

Collaboration and community of practice. Digital stories allow students to share ideas and knowledge. In 
Rambe and Mlambo’s (2014) study, students created a repository of digital stories in an attempt to address the 
articulation gap in South African higher education. The stories proved important to address skill deficiencies and 
help archive knowledge for other students. English foreign language students were able to use their classmates’ and 
teacher’s stories to build their own stories, and add them to a storytelling website (Tsou et ah, 2006). English 
language students collaborated to create digital stories, which resulted in improved listening comprehension, as 
compared to the control group who, “lacked similar opportunities for collaborative construction of meaning in 
authentic productive environments” (Yang & Wu, 2012, p. 346). Ranieri and Bruni (2013) found students were 
more involved in participating and sharing work when given the opportunity to create stories using mobile devices. 

Reflective thinking. Digital storytelling offers students an opportunity to reflect upon past experiences and 
delve into the reasons for actions (Kearney, 201 1). Sawyer and Willis (2011) used digital storytelling as a 
counseling method for high school students working with younger children. Pairs of students were able to identify 
potential issues and work through various endings, considering options. In addition, the permanency of digital 
storytelling enables students to review and reflect upon their creation (Kearney, 2011). Disadvantaged youth, with 
below average reading and writing abilities, collaborated to make digital stories to share with classmates. In the 
process of viewing peers’ digital stories, students were able to “compare their positions to those of their peers,” 
(Maier & Fisher, 2007, p. 181) as they engaged in personal reflection. 

Creative thinking. Students are inspired to think creatively, as a result of the need to research, document 
and produce digital stories. Digital stories are innately personal, a focal point that makes each story unique. In 
creating stories, students strive to make their work entertaining and original, even when dealing with historical or 
factual matters (Bou-Franch, 2012). Bran (2010) reported an increase in creativity after analyzing 78 digital stories 
created by journalism students. 

Building of digital literacy. The use of multimedia allows students to become familiar with creating digital 
content and navigating the Internet in a new way (Hartley, McWilliam, Burgess, & Banks, 2008). Di Bias and 
Paolini (2013) suggested the benefits of 20,000 students using a digital storytelling initiative included “the ability to 
decode, write through, and with, all forms of media, in specific with text, audio and images in multimedia form”(p. 
24). Pre-service teachers participated in a digital storytelling workshop and reported increased self-efficacy in 
regards to technology, which was then found to impact the comfort they felt with implementing technology in a 
classroom setting (Heo, 2009). The seventh grade students involved in Maier and Fisher’s (2007) work enjoyed 
learning how to create media pieces that were relevant to the media they consumed in their non-academic lives. 
Sadik’s (2008) study found digital storytelling offered Egyptian teenagers an opportunity to “acquire new media 
literacy and IT skills, including capturing and editing digital photos, recognizing different image formats, recording 
and using audio clips, searching the Web for text and images” (p. 502) for playback. 

Student engagement. As technology advances, students are able to share their stories with a greater 
audience. This inspires students to work harder on their projects. In addition, students tend to have an emotional 
connection with the stories they create, thus fostering greater engagement (Bran, 2010). Digital stories involve more 
than absorbing or reciting back information. There is a need for students to become actively involved in order to 
create the content. As a result, students must take an active role, rather than simply listen (Di Bias & Paolini, 2013). 
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Fails, Drain, and Guha (2010) noted that students were “engaged and energized by the prospeet of ereating 
something” (p. 2). Campbell (2012) measured the engagement of students writing digital stories eompared to stories 
written traditionally, and found a 26% inerease with student’s engagement based on time spent on task. Campbell 
also noted students pereeived themselves as better writers as a result of being able to ereate a tangible digital story. 
Reluetant readers were eneouraged to engage more in elassroom diseussions after being exposed to eontent via 
digital storytelling (Malin, 2010). A study of 32 ELL learners in Korea found digital storytelling helped students 
have a deeper understanding of the lesson, thus generating greater eonfidenee and engagement (Yoon, 2013). 

Quality of work. Digital storytelling leads to higher quality work for several reasons. First, students are able 
to work through drafts until they are satisfied with results (Campbell, 2012). Seeond, students are motivated to 
produee higher quality work in eonsideration of the potentially larger audienee when stories are posted online (Wake, 
2012). Third, students are often provided with seaffolding within storytelling software to help guide them in story 
ereation. For example, 114 third-graders were able to elaborate on their work and ereate more eomplex stories by 
using a eoneept map to ereate digital stories (Liu, Chen, Shih, Fluang & Liu, 2011). 

Challenges and issues regarding incorporation of digital storytelling 

Researchers identified several challenges when digital storytelling was used in the classroom. Several 
researchers cited access to technology as an issue, which created a problem integrating this method into instruction 
(Dogan & Robin, 2008; Dreon et ah, 2008; Flartley et ah, 2008; Sadik, 2008). In Wake’s (2012) study, 80 rural teens 
shared two computers while creating digital stories. While many of the students were connected to technology 
outside of the classroom, resources were scarce at school. In some studies, once connected, students and teachers 
became frustrated by technical problems (Choi & Johnson, 2005; Sadik, 2008). Heo (2009) noted that pre-service 
candidates showed a higher level of confidence with technology if they had used computers and the Internet for non- 
academic purposes. Ranieri and Brani (2013) experienced difficulties introducing a group of teens, aged 1 1-15, to 
digital storytelling because the teens were resistant to recording their voices, until given mobile phones to use. Also, 
some participants refused to be photographed, as a result of self-image issues. In this same study, publishing stories 
on Facebook was met with mixed results: to some participants, Facebook was viewed as a space for peer interaction 
and the digital stories were seen as a “threat to the [teens] reputation” (p. 226). 

Teachers and teacher candidates expressed concern in implementing digital storytelling due to the time 
involvement of planning and preparing lessons, advising students, and classroom management (Figg & McCartney, 
2010; Maier & Fisher 2007; Sadik, 2008). Teachers who were surveyed on barriers that kept them from using 
storytelling in the classroom ranked “time issues” high (Dogan & Robin, 2008). Teachers also expressed concerned 
about the quality of student work and the direct relationship between student produced stories and objectives of the 
subject matter (Sadik, 2008). Researchers warned that students’ creativity in producing digital stories could 
supersede learning objectives. Students may become more immersed in the development of the artifact than the 
educational content of the lesson (Chung, 2007; Hartley et ah, 2008; Kearney, 2011; van Gils, 2005). While Photo 
Story was frequently used in the studies (Heo, 2009; Hung, Hwang, & Huang, 2012; Leopold, 2010; Sadik, 2008; 
Wake, 2012; Yang & Wu, 2012), van Gils warned that formatted software could lead to dull stories. Effective 
teaching, in which teachers’ emphasized story development, was found key in terms of “going beyond the novelty or 
entertainment effect” (Campbell, 2012, p. 393) and increasing students’ writing skills. 

Instructional design suggestions 

Researchers suggested a structured plan for introduction, intervention, and reflection of digital storytelling 
(Chung, 2007; Di Bias, Garzotto, Paolini, & Sabiescu, 2009; Liu et ah, 2011; Sadik, 2008). Sandars, Murray, and 
Pellow (2008) found offering a step-by-step approach was important in creating successful stories in undergraduate 
medical education. To promote high quality writing, Figg and McCartney (2010) had student learners create a 
variety of digital stories that were sequenced in order of difficulty, so that the writing skills for the first digital story 
provided practice in basic skills for writing, and each subsequent story required the learner to build upon the skills 
learned in the previous task. Di Bias and Paolini (2013) offered six steps to structure digital stories: topic selection 
and gathering of material, content organization, content preparation, first version, evaluation, and final version. 

Kieler (2010) used the Center for Digital Storytelling’s seven elements of digital storytelling as a guide for her class 
and found each element to be important. When she unintentionally omitted three elements, she found her students’ 
stories to be disconnected. Researchers from the College of Education at the University of Houston offered 
instructional design guidelines after many years of focusing on digital storytelling (Robin & McNeil, 2012). This 
comprehensive list follows the ADDIE model of instructional design and includes analyzing the audience to develop 
an appropriate script, adding a personal connection, using a detailed script and storyboard, and understanding 
intellectual property issues. 
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It is important to offer support and examples. To guarantee the same lesson was delivered to all participants, 
Heo (2009) used a video tutorial that included instructions for how to create digital stories with Photo Story software. 
Instructors at Italian schools utilizing PoliCultura “made available online a sizable library of the works done by 
classes: this provided a strong drive for imitation” (Di Bias et ah, 2009, p. 16). Anchoring digital stories to real life 
makes the experience more motivating (Di Bias et ah, 2009; Maier & Fisher, 2007). Roby (2010) offered the 
content-related digital storytelling (CoRDS) model as a way to link subject matter with students in a personal way. 
This model helps instructors connect students to the subject manner, themselves and others. 

The method for creating digital storytelling should be considered, based on the equipment, technology and 
time available. Fails et al. (2010) suggested mobile devices appealed to children’s inclination to move and these 
devices inspired collaboration. Low cost, simplicity and the wide availability of authoring tools encouraged digital 
storytelling in a three-year experiment involving 381 classes (Di Bias et al., 2009). Kearney (2011) suggested 
instructors should consider outlets for publishing students’ digital story productions, from the general Internet to 
more specific school or district opportunities, as a way to showcase work. Psomos and Kordaki (2012) used the 
Dimension Star, a reference model to evaluate the effectiveness of digital storytelling environments. Kearney (2011) 
stressed the importance of carefully outlining the expectations for digital storytelling assignments. Sandars et al. 
(2008) emphasized the need to express the intention of the exercise to focus on reflection and deeper learning for 
students, over technical excellence. 


Discussion and Conclusion 

The educational value of digital storytelling is clear. Rather than simply adding technology to assignments, 
digital storytelling allows learners an opportunity to incorporate technology into classwork in a meaningful way, 
through the creation of artifacts that connect content in a personal manner. This is an effective method to increase 
engagement and develop a community of learners, as well as increase retention and inspire creativity. Unfortunately, 
studies are lacking regarding incorporating digital storytelling into online courses. In many ways online courses 
address the challenges involved with digital storytelling, overcoming issues associated with digital storytelling and 
technology. Students are already connected via technology to online courses and are largely proficient utilizing the 
media. Also, since time management in online learning is frequently tied to students’ interest, creating a digital story 
could be based on the time necessary for the student to complete the task, rather than a bell ringing in the classroom. 
Much of the success of digital storytelling involves effectively implementing it within courses, marrying content 
with students’ personal contributions. There must be a structured plan, and while many have been suggested, more 
research is necessary to determine if one structure is more effective for learning in a specific context. Providing 
students with examples can help guide production of stories and demonstrate the appropriate balance between 
content and creativity, as well as help inspire students. By having the stories published on the Internet, students are 
more likely to generate higher quality work. In online classes, digital stories can serve as particularly poignant 
artifacts that draw classmates together and build a community, based on the personal interjections that naturally 
evolve from the work. Digital stories also serve as powerful ways to give students a voice and build communication 
skills, aspects that bolster online course effectiveness. Digital storytelling offers a way to creatively involve students 
in online courses, generating greater engagement with course content and with other students, and producing quality 
results, aspects much needed to improve online education. 
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Introduction 

Technology allows information to be gathered by a push of a button or stroke on a keyboard. A world of 
information can be easily accessed from a phone that fits in the palm of one’s hand. Today, every aspect of life is 
flooded with images and diverse forms of media. The ability to “discriminate and interpret the visible actions, 
objects, symbols, natural or man-made ... to communicate with others” (Debes, 1969, p. 27), has become an 
increasingly important skill set for learners in and out of the classroom. The goal of this paper is to explore and 
identify the current trends of research pertaining to visual and media literacy. The researchers hope to show how 
research on visual and media literacy has evolved in the last fifteen years. 

Literature review 

Initially, the researchers explored past studies on visual and media literacy. Research-based articles 
focusing on visual and media literacy (VML) published between 2000 and 2014 were collected. The researchers 
identified a total of 129 articles to be included and reviewed for this study. Each article was reviewed for its content 
from 1) research question, 2) population studies, 3) methodologies, and 4) findings. 

Comparison of research questions 

The researchers identified similarities among research questions found in the 129 articles selected for the 
study. A common trend on the use of concept maps in education was identified. Twelve articles examined the 
classroom use of concept map and its effectiveness (Blunt & Karpickle, 2014; Chang & Chang, 2008; Harris & Zha, 
2013; Marculcu, Karakuyu & Dogan, 2013; Wahidin & Meerah, 2013). A majority of these articles found that visual 
component of concept maps helped facilitate learning. Another trend of inquiry focused on how learners use 
technology effectively in the classroom. Most articles looked into how students use technology in the classroom 
(Breman & Hassell, 2014; Dias & Trumpy, 2014; Lawanto, Santoso, Lawanto & Goodridge, 2014; Lee & So, 2014; 
Luckhardt, 2014). 

Several articles focused on how digital and media literacy could help improve students’ learning. This 
includes the influence of technology in the perception of popular media (Barden, 2014; Babad, Peer & Hobbs, 

2012), and enhanced understanding of content with the integration of technologies (Ashley, Lyden & Fasbinder, 
2012; Garcia-Ruiz, Ramierz-Garcia & Rodriguez-Rosell, 2014; Nakagaw & Arzubiaga, 2014; Schmidt, 2013; Sur, 
Unal & Iseri, 2014). A number of research articles focused on the connection between visual literacy and: 

1) Level of understanding of visual literacy among students (Asrlan & Nalinci, 2014; Brante, Olander & 
Nystrom, 2013; Brumberger, 2011; Jefferies, 2007; Mayall & Robinson, 2009); 

2) Improving learning abilities through visual literacy (Leigh, 2012; McTigue & Croix, 2010; Wang & 
Lee, 2014; Willis & Lockee, 2009; Yeh & Lohr, 2010); and 

3) Using various tools to improve visual literacy skills (Baker, 2010; Coleman, 2010; Hattwig, Bussert, 
Medaille & Burgess, 2013; Prior, Willson & Martinez, 2012; Sosa, 2009). 


37 



Other articles examined the use of visuals and demonstration of visual interpretation. Examples of visuals 
examined included graphic novels (Gavigan, 2011; Valerie & Abed, 2013), picture books (Maderazo, Martens, 
Croce, Martens, Doyle, Aghalarov & Noble, 2010; Yu, 2009), and other images (Farha, 2009; Jin & Boling, 2010; 
Kovalik & Williams, 2011; Mahmood & Femely, 2006). These research articles examined how students’ interpret 
visual content to facilitate better communication. 

Comparison of population studied 

The articles reviewed covered different populations. At least four different populations or sources of data 
have been identified, including 

1) Educators (Craig, 2013; Ismail, Bokhare, Azizan & Azman, 2013; Lane, 2013; Nakagawa & 
Arzubiaga, 2014; Parkhill & Davey, 2014), 

2) General population (Boyle & Cook, 2001; Brown, Hardaker & Higgett, 2000; Brumberger, 2011; 
Newfield, 2011), 

3) Students including undergraduates (Barden, 2014; Dias & Trumpy, 2014; Garcia-Ruiz, Ramirez- 
Garcia & Rodriguez-Rosell, 2014; Kupczynski, Brown, Holland & Uriegas, 2014; Lawanto, Santoso, 
Lawanto & Goodridge, 2014); graduate (Arslan & Nalinci, 2014; Brante, Olander & Nystrom, 2013; 
Getzlaf, Perry, Toffner, Lamarch & Edwards, 2009; Stewart-McKoy, 2014; Sutton, 2014); elementary 
(Arend, 2009; Hattwig, Bussert, Medaille & Burgess, 2013; Heilmann, 2012; Jin & Boling, 2010; 

Pinto, 2014); middle school (Baker, 2010; Luckhardt, 2014; Percival & Percival, 2009; Power, 2008; 
Valerie & Abed, 2013); and high school (Ashley, Lyden & Fasbinder, 2012; Gold, 2004; Harris & 

Zha, 2013; Hilbert & Renkl, 2008; Jones & Cooke, 2006), and 

4) Articles from past studies (Breman & Hassell, 2014; Coleman, 2010; Efaw, Hampton, Martinez & 
Smith, 2004; Kurtz, Beaudoin & Sagee, 2004; Mayall & Robinson, 2009). 

Comparison of methodologies 

Two types of methodologies were used for gathering the data: qualitative and quantitative. Qualitative data 
collection involved: comparison (Ke, Lin, Ching & Dwyer, 2006; Newfield, 2011; Parkhill & Davey, 2014; Schwarz 
& Crenshaw, 2011; Sutton, 2014), discussion (Baker, 2013; Luckhardt, 2014; Nakagawa & Arzubiaga, 2014; Rye, 
Landenberger & Warner, 2013; Wichadee, 2014), interviews (Heilmann, 2012; Schulte, 2010; Sur, Unal & Iseri, 
2014; Redmond, 2012; Valerie & Abed, 2013), and observations (Barden, 2014; Gavigan, 2011; Kenny, 2011; Lee 
& So, 2014; Young, 2012). The data collected were coded and organized for content analysis by cluster or 
comparison. 

The research articles used generated data using the following collection strategies: 1) online surveys 
(Heilmann, 2012; Sur, Unal & Iseri, 2014), 2) Likert scale surveys (Dinet, Marquet & Nissen, 2003; Gavigan, 2011; 
Grzeda & Miller, 2009; Wichadee, 2014), 3) pre- and post- test (Kenny, 2011; Valerie& Abed, 2013), and 4) open- 
ended questionnaire (Arend, 2009; Hilbert & Renkl, 2008; Freeman & Jessup, 2004; Scull & Kupersmidt, 2010; 
Shaikh & Macaulay, 2001). Surveys were often collected online and had both open and closed questions. 

Comparison of findings 

Four common themes were identified in the findings, including (a) the need for teaching VML (Garcia- 
Ruiz, Ramirez-Garcia & Rodriguez-Rosell, 2014; Marulcu, Karakuyu & Dogan, 2013; Redmond, 2012; Schmidt, 
2013; Sur, Unal & Iseri, 2014);(b) the use of visuals and media helps to facilitate critical thinking (Blunt & 
Karpickle, 2014; Harris & Zha, 2013; Parkhill & Davey; 2014; Rye, Landenberger & Warner, 2013; Schwarz & 
Crenshaw, 2011); (c) Visuals can improve a student’s performance (Chang & Chang, 2008; Ke, Lin, Ching & 
Dwyer, 2006; Nesbit & Adesope, 2006; Percival & Percival, 2009; Wahidin & Meerah, 2013); and (d) Visuals can 
motivate students (Barden, 2014; Gavigan, 2011; Kenny, 2011; Leigh, 2012; Valerie & Abed, 2013). 

All articles found for this exploratory study support the use of visuals and media in the classroom. Most 
studies supported the idea that VML is important in furthering the development of technology-supported classrooms. 
Similarly, Sosa (2009) argued in her work that “the results of this study indicate that visual literacy truly is the 
missing piece of many technology integration courses” (p. X). These articles as sources of data for our study will 
help in exploring how VML research changed since 2000 and will better informed the directions of current and 
future research practices and needs. 
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Methods 


This paper is an exploratory study of research-based articles pertaining to VML published between the 
years 2000 and 2014. The purpose of this paper is to show how VML is currently being researched and how that 
research has changed in the past fifteen years. The sources of data used included a population of 129 peer-reviewed 
articles collected from forty-one journals. These articles were published between the years 2000 and 2014. The 
primary method in collecting these peer-reviewed articles involved searching the online databases and journal 
archives. Printed copies of peer-reviewed journal articles provided by peers and colleagues were also used. Articles 
found meeting the stated requirements were printed or pulled, and later organized for analysis. 

In this exploratory study, the researchers reviewed each article included in the study. An article’s content 
was coded using four categories: (a) research question, (b) population studied, (c) methodologies, and (d) findings. 
For analysis, similarities within categories were identified as well as relationships between categories. Qualitative 
data was quantified to better facilitate a comparative approach among the articles. Using an Excel spreadsheet, the 
articles’ content was compared using the following indicators: a) date of publication, b) journal title, c) key words, 
d) location of the study, and e) primary author(s). 


Findings 

The data analysis generated the following findings: 1) peer-reviewed journals publishing visual and media 
literacy research, 2) research article productivity by publication year, 3) top keywords in peer-reviewed articles 
related to VML research, 4) relationship journals and keywords, and 

Journal publications 

Four peer-reviewed journals published more research articles directly or indirectly related to VML — 
Journal of Educators Online (JEO), Research in Learning Technology (RET), Journal of Visual Literacy (JVL), and 
Journal of Media Literacy Education (JMLE). Table 1 illustrates the relationship of the number of VML research 
articles published in a journal and publication year. Out of the four top journals, RLT, started publishing VML 
research articles in 2000 but has barely published one in the last five years. However, the journal contributed about 
17% of articles used in this study. The JEO has increased its frequency of publishing VML research since 2004. This 
journal supplied forty research-based articles related to VML research, totaling about 30% of all the articles used in 
this study. 

JVL was the third highest publisher of research-based articles relating to VML research. However, this 
journal only published 1 1 research-based articles. The researchers expected more research-based articles but 
discovered that, though it had published extensively on visual and media literacy, the majority of the articles focused 
on defining the VML as a field of study, and less on research. 
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Table 1. Frequency of VML Article Publication from 2000-2014 by Peer-Reviewed Journals 



Y ear of Publication 


Journal 

14 

13 

12 

11 

10 

09 

08 

07 

06 

05 

04 

03 

02 

01 

00 

Tot 

al 

Journal of Educators 
Online (JEO) 

10 

6 

2 

2 

2 

4 

2 

2 

3 

4 

3 





40 

Research in Learning 
Technology (RLT) 

1 

— 

— 

— 

" 

1 

" 

1 

2 

" 

2 

3 

3 

6 

4 

23 

Journal of Visual Literacy 
(JVL) 


1 


2 

4 

1 


2 

1 







11 

Journal of Media Literacy 
Education (JMLE) 

— 

1 

3 

2 

2 

" 

" 

" 

" 

" 

" 

" 

" 

" 

" 

8 

Communicar 

3 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

3 

Merlot Journal of Online 
Learning & Teaching 
(JOLT) 

- 

- 

- 

- 

1 

- 

1 

1 

- 

- 

- 

- 

- 

- 

- 

3 

EDUCAUSE Quarterly 

- 

— 

— 

1 

— 

— 

— 

— 

— 

— 

1 

— 

— 

— 

— 

2 

Instructional Science (IS) 

— 

— 

— 

— 

— 

— 

1 

- 

— 

— 

— 

— 

— 

— 

1 

2 

Journal of Computer- 
Assisted Learning (JCAL) 

— 

— 

— 

— 

" 

" 

" 

" 

" 

" 

" 

1 

" 

1 

" 

2 

Language Arts 

— 

— 

1 

— 

1 

- 

— 

— 

— 

— 

— 

— 

— 

— 

— 

2 

TechTrends 

— 

— 

— 

— 

— 

2 

— 

— 

— 

— 

— 

— 

— 

— 

— 

2 

Turkish Online Journal of 
Educational Technology 
(TOJET) 

2 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

2 


Further, the overall review of the data collected revealed that the publication of VML research continued to 
increase after 2008. Table 2 illustrates the number of VML research articles published annually. Most articles were 
published in 2010, 2013, and 2014. This fact helps support the earlier statement that research on VML has been 
slowly increasing, especially within the last seven years. 

Table 2. Frequency of Article Published on VML Research by Publication Year 



Year of Publication 


14 

13 

12 

11 

10 

09 

08 

07 

06 

05 

04 

03 

02 

01 

00 

Total 

Number of VML 
Research Articles 
Published 

21 

15 

9 

9 

12 

9 

5 

7 

7 

4 

9 

7 

3 

7 

5 

129 


Journal productivity 


The researchers identified 2-3 keywords based on the content of the article. Table 3 illustrates the range of 
keywords identified by the researchers from the articles in the span of fifteen years. In completing this task, the 
researchers used the keywords identified by the author(s) in the articles. For those articles with no keywords 
identified, the researchers generated some by determining the article’s main focus. For example, if an article was 
researching on the effectiveness of concept maps, then researchers would choose “concept maps” as one of the 
keywords. 
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Table 3. Top keywords in journals that publish VML research articles by year of publication 



Year of Publication 


Keyword 

14 

13 

12 

11 

10 

09 

08 

07 

06 

05 

04 

03 

02 

01 

00 

Total 

Online Education 

10 

5 

1 

2 

1 

3 

1 

2 

4 

5 

3 

2 

2 

3 

3 

47 

Visual Literacy 

2 

3 

3 

2 

9 

3 

— 

3 

1 

— 

— 

1 

— 

— 

— 

27 

Technology Integration 

2 

1 

— 

1 

2 

5 

— 

— 

— 

— 

2 

2 

1 

3 

2 

21 

Media Literacy 

3 

1 

3 

2 

3 

4 

— 

— 

— 

— 

— 

2 

— 

— 

— 

18 

Concept Maps 

1 

4 

- 

- 

- 

- 

2 

1 

1 

- 

2 

- 

- 

- 

- 

12 


Keywords 


The top five keywords identified in selected VML research articles were Concept Maps, Media Literacy, 
Online Education, Technology Integration, and Visual Literacy. In the span of 15 years, “Online Education” was the 
top keyword identified from 47 VML research articles. “Visual Literacy” was the next frequently mentioned 
keyword. Most of the research on visual literacy focused on how well students and teachers understand and interpret 
visuals. The majority of research on this topic tried to determine if visual literacy is being taught and understood 
well within classrooms. Table 3 illustrates the trends of keywords as well as demonstrates how research activities on 
related topics seem to have increased since 2000. 

“T echnology Integration” is another keyword that researchers found heavily mentioned in VML research 
articles. However, a gap spanning from 2005 until 2008 seems evident and could be attributed to shifting ideas or 
research activities over time. However, even with this gap in research activities, the researchers identified twenty- 
one articles that studied how technology is being used in education. The majority of research articles examined the 
types of technologies being used in the classroom and their effectiveness. 

Other top keywords included “Media Literacy” and “Concept Maps”. Most of the research activities 
focusing on media literacy occurred after 2008 while for concept maps, many activities occurred between 2006 and 
2008. Table 3 also showed that trends of VML research activities have changed since 2000. Online Education is the 
only keyword that was able to demonstrate some staying power — slow but increasing popularity since the early 
2000s. 

The researchers were interested in examining if there is a relationship between the top ten journals and the 
top five keywords. In using the top keywords from Table 3, researchers connected the types of research projects 
undertaken for the last 15 years, and published in top journals. The top ten journals are JEO, RET, JVL, JMLE, 
Communicar, Merlot JOLT, EDUCAUSE Quarterly, JCAL, Language Arts, and Instructional Science. Cumulative 
totals from the study were posted in Table 4. 

Table 4. Relationship between journals publishing VML research articles and keywords 



Top Journals Publishing VML Research Articles by Keywords 

Keywords 

JEO 

RLT 

JVL 

JMLE 

Communicar 

Merlot 

JOLT 

EDUCAUSE 

Quarterly 

JCAL 

LA 

IS 

Online 

Education 

32 

10 

— 

— 

— 

— 

— 

1 

— 

1 

Visual Literacy 

1 

— 

9 

— 

— 

— 

— 

— 

1 

— 

Technology 

Integration 

2 

9 

— 

— 

— 

— 

2 

— 

" 

" 

Media Literacy 

— 

— 

— 

8 

2 

— 

— 

— 

— 

— 

Concept Maps 

— 

— 

- 

— 

— 

— 

— 

1 

— 

1 

TOTAL 

35 

19 

9 

8 

2 

0 

2 

2 

1 

2 


Journal productivity and keywords 


“Online Education” was the top keyword found among 44 articles (34.1%) and reflected a trend in visual 
and media literacy research. JEO and RLT were the two journals that published more VML research articles with 
this keyword in the last 15 years. JEO published 32 (24.8%) articles while RLT had 10 (7.75%). 
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“Visual literacy” was the second most common keyword identified from the study. JVL published nine 
(6.9%) articles with this keyword given the same timeline. RLT published nine (6.9%) articles for “technology 
integration” keyword while JMLE had eight (6.2%) for “media literacy”. 

Further, with the number of articles identified for this study, the researchers wanted to know where 
research activities are undertaken and who are authoring them. Table 5 provided information on the location of the 
VML research studies. Researchers examined articles published in the United States and Canada separately. Europe 
was also scrutinized and identified which countries produce VML studies. 

Table 5. Region or country where the research on visual and media literacy was completed 


Region or 
Country 

United 

States 

Caribbean 

Region 

Canada 

Europe 

Asia 

Africa 

Australia 

TOTAL 

TOTAL 

81 

1 

6 

28 

10 

1 

2 

129 


Authorship 

In identifying the authors of the articles, the researchers expected to find multiple articles published by the 
same author. Flowever, the results informed the researchers that most authors published only once as a primary or 
lead author. There were a small number of cases where a primary author served as secondary author for another 
article, but for the most part they only published once. Only one author was found to have published twice as the 
primary author. It seems that the lack of repeat publications of similar if not related research articles by an author 
indicate a potential lack of interest in researching and publishing in VML. 

Region or country 

Two regions or countries have been identified as places where VML research was conducted: United States 
and Europe. A majority of research studies took place in the United States. Eighty-one studies (60%) were 
completed in America. The second region or country that produced a great number of studies is Europe with 28 
(20%) studies on visual and media literacy. Most studies done in Europe were from the United Kingdom. Asia was 
the third highest publishing region with ten studies. Six published studies originated from Canada and fwo from 
Ausfralia. One article each came from Africa and the Caribbean. 

Most research studies took place in the United States and the United Kingdom. This fact was not surprising 
since both countries are technologically advanced compared to other. So it is expected that both would be leading 
research activities focusing on VML. 


Implications 

This exploratory study shows that research on VML is lacking if not minimal in the last 15 years. Flowever, 
there is an indication that research interest in this area has been slowly growing. In reviewing the content of articles 
published in early 2000s, it seems there is a shift among authors to research more on the value, benefit or impact of 
VML rather than simply trying to define the field. This study also shows that research on VML is interdisciplinary 
or multidisciplinary. An example would be the integration of VML with online education or technology integration. 
While these questions do not always pertain to visual and media literacy, it was observed that a large amount of data 
gathered sought to answer questions that pertained to those subjects. 

This study also shows that research on VML is taking place over a wide range of disciplines, not simply 
education or mass communication. Several of the journals have articles pertaining to other disciplines, such as, 
science, language arts, and history. What this means is that research on VML encompasses multiple fields of study. 
Finally, VML research studies are taking place all over the world. Flowever, the majority of VML studies are 
completed in the United States and Europe. 
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Limitations and Recommendations 


Given the exploratory nature of this study, the researehers are aware of several limitations. One hundred 
twenty-nine artieles on VML were eolleeted for this study using online databases, journal arehives, and printed 
materials. Some artieles may not have been available or aeeessible to the researehers given the limited time and 
resourees. This eould have impaeted the analysis and interpretation of the results for making any generalization. 
Given the number of researeh artieles from the United States and Europe and the limited amount from Asia or other 
regions of the world, the researehers believe there is a potential of bias towards Western thinking for this field of 
study. Also, researehers reeommend the identifieation of more artieles to be ineluded in a follow up study espeeially 
from researeh aetivities eompleted beyond North Ameriea and Europe. 

Conclusions 

This exploratory study identified eurrent trends of VML researeh. The findings provided information on 
journals publishing VML researeh ineluding inereased produetivity from 2000 to 2014. Also, the findings 
demonstrated the interdiseiplinary or multidiseiplinary nature of VML researeh, and the inereased use and 
integration of teehnology, espeeially in online edueation, that eontinues to support VML researeh. Visuals and 
media will eontinuous influenee or impaet everyday life. This makes VML researeh important to those who teaeh 
how to interpret and use visuals and media in order to eommunieate with others. VML knowledge and skills are 
neeessity to those engage in all levels of edueation. 


References 

Arend, B. (2009). Encouraging critical thinking in online threaded discussions. The Journal of Educators Online, 
6(\), 1-23. 

Arslan, R., & Nalinci, G. Z. (2014). Development of visual literacy levels scale in higher education. The Turkish 
Online Journal of Educational Technology, 13(2), 61-70. 

Ashley, S., Lyden, G., & Fasbinder, D. (2012). Exploring message meaning: A qualitative media literacy study of 
college freshmen. Journal of Media Literacy Education, 4(3), 229-243. 

Babad, E., Peer, E., & Flobbs, R. (2012). Media literacy and media bias: Are media literacy students less susceptible 
to nonverbal judgment biases? Psychology of Popular Media Culture, 7(2), 97-107. 

Baker, C. (2010). The impact of instructor immediacy and presence for online student affective learning, cognition, 
and motivation. The Journal of Educators Online, 7(1), 1-30. 

Baker, F.W. (2013). Reading visual media. MiddleWeb Guest Article. 

Barden, O. (2014). Facebook levels the playing field: Dyslexic students learning through digital literacies. Research 
in Learning Technology, 22, 1-18. 

Bender, D. M. (2005). Developing a collaborative multidisciplinary online design course. The Journal of Educators 
Online, 2(2), 1-12. 

Berman, R., & Flassell, D. (2014). Digital native and digital immigrant use of scholarly network for doctoral 
learners. The Journal of Educators Online, 77(1), 1-26. 

Blake, C. T., & Scanlon, E. (2003). Enriching accounts of computer-supported collaboration by using video data. 
Research in Learning Technology, 77(2), 5-13. 

Blunt, J. R., & Karpickle, J. D. (2014). Learning with retrieval-based concept mapping. Journal of Educational 
Psychology, 1-10. 

Boyle, T., & Cook J. (2001). Online interactivity: Best practice based on two case studies. Research in Learning 
Technology, 9(1), 94-102. 

Brante, E.W., Olander, M. FI., & Nystrom, M. (2013). Exploring the impact of contrasting cases in text and picture 
processing. Journal of Visual Literacy, 32(2), 15-38. 

Brill, J. M., Kim, D., & Branch, R. M. (2007). Visual literacy defined-The results of a Delphi study: Can IVLA 
(operationally) define visual literacy? Journal of Visual Literacy, 27(1), 47-60. 

Brown, S., Flardaker, C. FI. M., & Fliggett, N. P. (2000). Designs on the Web: A case study of online learning for 
design students. Research in Learning Technology, 5(1), 30-40. 

Brumberger, E. (2011). Visual literacy and the digital native: An examination of the millennial learner. Journal of 
Visual Literacy, 30(\), 19-46. 

Buchanan, E. A., Myers, S. E., & Flardin, S. L. (2005). Flolding your hand from a distance: Online mentoring and 
the graduate library and information science student. The Journal of Educators Online, 2(2), 1-18. 


43 



Butterfield, A., Sutherland, R., & Molyneux-Hodgson, S. (2000). Learning conversations in science: The case of 
vocational students in the UK. Research in Learning Technology, cS(3), 89-104. 

Chang, K. E., Sung, Y. T., & Chen, S. F. (2001). Learning through computer-based concept mapping with 
scaffolding aid. Journal of Computer Assisted Learning, 77, 21-33. 

Chang, S. L., & Chang Y. (2008). Using online concept mapping with peer learning to enhance concept application. 
The Quarterly Review of Distance Education, 9(1), 17-27. 

Chen, C. C., & Jones, K. T. (2007). Blended learning vs. traditional classroom settings: Assessing effectiveness and 
student perceptions in a n MBA accounting course. The Journal of Educators Online, 4{\), 1-15. 

Chularut, P., & DeBaker, T. K. (2004). The influence of concept mapping on achievement, self-regulation, and self- 
efficiency in students of English as a second language. Contemporary Educational Psychology, 29, 248- 
263. 

Clarke, L. W., & Besnoy, K. D. (2010). Connecting the old to the new: What technology-crazed adolescents tell us 
about teaching content area literacy. Journal of Media Literacy Education, 2(1), 47-56. 

Coleman, J. (2010). Elementary teachers’ instructional practices involving graphical representations. Journal of 
Visual Literacy, 29(2), 199-222. 

Collis, B., & Messing, J. (2001). Usage, attitudes and workload implications for a web-based learning environment. 
Research in Learning Technology, 9(1), 17-25. 

Cordova, J. L. (2003). The use of web-based visualization techniques and its effect on student comprehension. 
Journal of computing Sciences in Colleges, 18(6), 67-71. 

Craig, D. V. (2013). Content creators and language learners: Exploring Web 2.0 and wikis. The Journal of 
Educators Online, 10(1), 1-21. 

De Freitas, S., & Roberts, G. P. (2003). Does distance e-leaming work? A comparison between distance and face-to- 
face learners using e-leaming materials. Research in Learning 

Debes, J. (1969). The loom of visual literacy: An overview. Audiovisual Instruction, 74(8), 25-27. 

DePriter, T. (2013). Ta investigation of teaching strategy in the distance learning mathematics classroom. The 
Journal of Educators Online, 10(1), 1-20. 

Dias, L. P., & Trampy, R. (2014). Online instmctor’s use of audio feedback to increase social presence and student 
satisfaction. The Journal of Educators Online, 77(2), 1-19. 

Dinet, J., Marquet, P., & Nissen, E. (2003). An exploratory study of adolescent’s perceptions of the web. Journal of 
Computer Assisted Learning, 19, 538-545. 

Dobemig, K., Lobinger, K., & Wetzstein, I. (2010). Covering conflict: Differences in visual and verbal news 

coverage of the Gaza crisis 2009 in four weekly news media. Journal of Visual Literacy, 29(1), 88-105. 

Dron, J., Seidel, C., & Litten, G. (2004). Transactional distance in a blended learning environment. Research in 
Learning Technology, 72(2), 163-174. 

Duffy, T., Gilbert, L, Kennedy, D., & Kwong, P. W. (2002). Comparing distance education and conventional 
education: Observations from a comparative study of post-registration nurses. Research in Learning 
Technology, 79(1), 70-81. 

Efaw, J., Hampton, S., Martinez, A., & Smith, S. (2004). Miracle or menace: Teaching and learning with laptop 
computers in the classroom. Educause Quarterly, 10-18. 

Emery, W., & McCabe, R. (2003). The development of media literacy among grade five teachers and students- a 
case study. Canadian Journal of Learning and Technology, 29(1), 1-25. 

Farha, N. W. (2009). An exploratory study into the efficiency of learning objects. The Journal of Educators Online, 
6(2), 1-32. 

Freeman, L. A., & Jessup, L. M. (2004). The power and benefits of concept mapping: Measuring use, usefulness, 
ease of use, and satisfaction. International Journal of Science Education, 26(2), 151-169. 

French, F., Cumpson, L, & Wood, R. (2001). The specification and design of an interactive virtual environment for 
use in teacher training. Research in Learning Technology, 9(1), 50-61. 

Gainer, J. S. (2010). Critical media literacy in middle school: Exploring the politics of representation. Journal of 
Adolescent and Adult Literacy, 55(5), 364-373. 

Garcia-Ruiz, R., Ramirez-Garcia, A., & Rodriguez-Rosell, M. M. (2014). Media literacy education for a new 
prosumer citizenship. Comunicar, 22(43), 15-23. 

Garrett, B. M., & Callear, D. Designing intelligent computer-based simulations: A pragmatic approach. Research in 
Learning Technology, 9(3), 5-16. 

Gavigan, K. (2011). More powerful than a locomotive: Using graphic novels to motivate struggling male adolescent 
readers. The Journal of Research on Libraries and Young Adults, 7(3), 1-11. 


44 



Gergle, D., Kraut, R. E., & Fussell, S. R. (2004). Language efficiency and visual technology minimizing 

collaborative effort with visual information. Journal of Language and Social Psychology, 23(4), 491-517. 

Getzlaf, B., Perry, B., Toffner, G., Lamarche, K., & Edwards, M. (2009). Effective instructor feedback: Perceptions 
of online graduate students. The Journal of Educators Online, 6(2), 1-22. 

Gold, S. S. (2004). The effect of software facilitated communication on student outcomes in online classes. The 
Journal of Educators Online, 7(1), 1-15. 

Grzeda, M., & Miller, G. E. (2009). The effectiveness of an online MBA program in meeting mid-career student 
expectations. The Journal of Educators Online, 6(2), 1-16. 

Hachey, A. C., Wladis, C. W., & Conway, K. M. (2012). Is the second time the charm? Investigating trends in 
online re-enrollment, retention and success. The Journal of Educators Online, 9(1), 1-25. 

Hara, N., Bonk, C. J., & Angeli, C. (2000). Content analysis of online discussion in an applied educational 
psychology course. Instructional Science, 28, 115-152. 

Harrington, C. F., & Reasons, S. G. (2005). Online student evaluation of teaching for distance education: A perfect 
match? The Journal of Educators Online, 2(1), 1-12. 

Harris, C. M., & Zha, S. (2013). Concept mapping: A critical thinking technique. Education, 134(2), 207-211. 

Hattwig, D., Bussert, K., Medaille, A., & Burgess, J. (2013). Visual literacy standards in higher education: New 
opportunities for libraries and student learning. Libraries and the Academy, 75(1), 61-89. 

Hay, D. B. (2007). Using concept maps to measure deep, surface and non-leaming outcomes. Studies in Higher 
Education, 52(1), 39-57. 

Heilmann, S. G. (2012). Can you hear me now? Assessing students’ classroom communication preferences via a 
telephone conference activity. The Journal of Educators Online, 9(1), 1-15. 

Hilbert, T. S., & Renkl, A. (2008). Concept mapping as a follow-up strategy to learning from texts: what 
characterizes good and poor mappers? Instructional Science, 36, 53-73. 

Ismail, I., Bokhare, S. F., Azizan, S. N., & Azman, N. (2013). Teaching via mobile phone: A case study on 

Malaysian teachers’ technology acceptance and readiness. The Journal of Educators Online, 79(1), 1-38. 

Jameson, J., & Squires, D. (2000). Teaching new media composition studies in a lifelong learning context. Research 
in Learning Technology, <5(3), 41-55. 

Jefferies, E. (2007). Devising a method for improving design education of digital visual skills. Journal of Visual 
Literacy, 27(1), 123-138. 

Jin, S. H., & Bolling, E. (2010). Instructional designers’ intentions and learners’ perceptions of the instractional 
functions of visuals in an e-leaming context. The Journal of Visual Literacy, 29(2), 143-166. 

Joiner, R., Durkin, C., Morrison, D., & Williams, L. (2003). Activating boxmind: An evaluation of a web-based 
video lecture with synchronized activities. Research in Learning Technology, 77(3), 19-30. 

Jones, A., Barnard, J., Calder, J., Scanlon, E., & Thompson, J. (2000). Evaluating learning and teaching technologies 
in further education. Research in Learning Technology, <5(3), 56-66. 

Ke, F., Lin, H., Ching, Y. H., & Dwyer, F. (2006). Effects of animation on multi-level learning outcomes for 
learners with different characteristics: A meta-analytic assessment and interpretation. Journal of Visual 
Literacy, 26(1), 15-40. 

Kenny, R. F., (2011). Beyond the Gutenberg parenthesis: Exploring new paradigms in media learning. Journal of 
Media Literacy Education, 5(1), 32-46. 

Kerr, M. S., Rynearson, K. & Kerr, M. C. (2004). Innovative educational practice: Using virtual labs in the 
secondary classroom. The Journal of Educators Online, 7(1), 1-9. 

King, S. E., & Arnold, K. C. (2012). Blended learning environments in higher education: A case study of how 
professors make it happen. Mid-Western Educational Researcher, 25(1), 44- 59. 

Kovalik, C. L., & Williams, M. A. (2011). Cartoons as advance organizers. Journal of Visual Literacy, 30(2), 40-64. 

Kupczynski, L., Brown, M., Holland, G., & Uriegas, B. (2014). The relationship between gender and academic 
success online. The Journal of Educators Online, 77(1), 1-14. 

Kurtz, G., Beaudoin, M., & Sagee, R. (2004). From campus to web: The changing roles of faculty from classroom to 
online teaching. The Journal of Educators Online, 7(1), 1-28. 

Lane, L. M. (2013). An open, online class to prepare faculty to teach online. The Journal of Educators Online, 

10(1), 1-32. 

Lapsley, R., Kulik, B., Moody, R., & Arbaugh, J. B. (200). Is identical really identical? An investigation of 
equivalency theory and online learning. The Journal of Educators Online, 5(1), 1-19. 

Lawanto, O., Santoso, H. B., Lawanto, K. N., & Goodridge, W. (2014). Self-regulated learning skills and online 

activates between higher and lower performances on a web intensive undergraduate engineering course. The 
Journal of Educators Online, 77(3), 1-32. 


45 



Lee, A.Y. L., & So, C. Y. K. (2014). Media literaey and information literaey: Similarities and differenees. 
Comunicar, 27(42), 137-145. 

Leigh R. S. (2012). Writers draw visual hooks: Children’s inquiry into writing. Language Arts, 89{6), 396-404. 

Loekyer, L., Patterson, J., Rowland, G., & Heame, D. (2002). Online mentoring and peer support: Using learning 

teehnologies to faeilitate entry into a eommunity of praetiee. Research in Learning Technology, 70(1), 24-31. 

Loveland, K. A. (2007). Student evaluation of teaehing (SET) in web-based elasses: Preliminary findings and a eall 
for further researeh. The Journal of Educators Online, 4(2), 1-18. 

Luekhardt, C. (2014). Teaehing historieal literaey and making world history relevant in the online diseussion board. 
History Teacher, 47(2), 187-196. 

Lou, H., Robinson, A. C., & Detwiler, J. (2014). Effeet of geographie distanee on distanee edueation: An empirieal 
study. The Journal of Educators Online, 77(3), 1-26. 

Maderazo, C., Martens, P., Croee, K., Martens, R., Doyle, M., Aghalarov, S., & Noble, R. (2010). Beyond pieture 
walks: Revaluing pieture books as written and pietorial text. Language Arts, 87(6), 437-446. 

Mahmood, K. A., & Femely, E. (2006). The use of animated agents in e-leaming environments: An exploratory, 
interpretive ease study. Research in Learning Technology, 14(2), 153-168. 

Mandemaeh, J. B. (2005). Relative effeetiveness of eomputer-based and human feedbaek for enhaneing student 
learning. The Journal of Educators Online, 2(1), 1-17. 

Mandemaeh, J. B., Fludson, S., & Wise, S. (2013). Where has the time gone? Faeulty aetivities and time 
eommitments in the online elassroom. The Journal of Educators Online, 10(2), 1-15. 

Mamleu, L, Karakuyu, Y., & Dogan, M. (2013). Can elementary students gather information from eoneept maps? 
International Journal of Environmental and Science Education, 5(1), 611-625. 

Mayall, Ft. J., & Robinson, R. S. (2009). Investigating visual literaey integration: Linda’s legaey? TechTrends: 
Linking Research and Practice to Improve Learning, 53(2), 48-49. 

MeTigue, E., & Croix, A. (2010). Visual Literaey in Seienee. Science Scope, 55(9), 17-22. 

Mraz, M., Heron, A. H., & Wood, K. (2003). Media literaey, popular eulture, and the transfer of higher order 
thinking abilities. Middle School Journal, 54(3), 51-56. 

Nakagawa, K., & Arzubiaga, A. E. (2014). The use of soeial media in teaehing raee. Adult Learning, 25(3), 103-110. 

Nesbit, J. C., & Adesope, O. O. (2006). Learning with eoneept and knowledge maps: A meta-analysis. Review of 
Educational Research, 76(3), 413-448. 

Newfield, D. (2011). From visual literaey to eritieal visual literaey: An analysis of edueational materials. English 
Teaching: Practice and Critique, 70(1), 81-94. 

Park, Y. J., & Bonk, C. J. (2007). Is online life a breeze? A ease study for promoting synehronous learning in a 
blended graduate eourse. Merlot Journal of Online Learning and Teaching, 5(3), 307-323. 

Parkhill, F., & Davey, R. (2014). T used to read one page in two minutes now I am reading ten’: Using popular fdm 
subtitles to enhanee literaey outeomes. Australian Journal of Language and Literacy, 57(2), 28-33. 

Patron, H., & Lopez, S. (2011). Student effort, eonsisteney, and online performanee. The Journal of Educators 
Online, 5(2), 1-11. 

Pearson, E. J., & Koppi, T. (2002) Inelusion and online learning opportunities: Designing for aeeessibility. Research 
and Learning Technology, 10(2), 17-28. 

Peng, Z. (2006). Applying internet-based teehnologies to teaehing eorporate finanee and investments. The Journal of 
Educators Online, 5(1), 1-15. 

Pereival, J., & Pereival, N. (2009). A ease of a laptop learning eampus: How do teehnology ehoiees affeet 
pereeptions?. Research in Learning Technology, 77(3), 173-186. 

Peraski, L., & Mishra, P. (2004). Webs of aetivity in online eourse design and teaehing. Research in Learning 
Technology, 72(1), 37-49. 

Pinto, M. B. (2014). The use of Yammer in higher edueation: An exploratory study. The Journal of Educators 
Online, 77(1), 1-33. 

Power, M. (2008). The emergenee of a blended online learning environment. Merlot Journal of Online Learning and 
Teaching, 4(4), 502-514. 

Prior, L. A., Willson, A., & Martinez, M. (2012). Pieture this: Visual literaey as a pathway to eharaeter 
understanding. The Reading Teacher, 66(3), 195-206. 

Redmond, T. A. (2012). The pedagogy of eritieal enjoyment: Teaehing and reading the hearts and minds of 

adoleseent learners through media literaey edueation. Journal of Media Literacy Education, 4(2), 106-120. 

Russo-Gelieher, R. J. (2013). Qualitative insights into faeulty use of student supported serviees with online students 
at risk: Implieations for student retention. The Journal of Educators Online, 10(1), 1-32. 


46 



Rye, J., Landenberger, R., & Warner, T. A. (2013). Incorporating concept mapping in project-based learning: 
Lessons from Watershed Investigation. Journal of Science Education and Technology, 22(3), 379-392. 

Schmidt, H. C. (2013). Media literacy education from kindergarten to college: A comparison of how media 
literacy is addressed across the educational system. Journal of Media Literacy Education, 5(1), 295-309. 

Schulte, M. (2010). Faculty perceptions of technology distance education transactions: Qualitative outcome to 
inform teaching practices. The Journal of Educators Online, 7(2), 1-34. 

Schwarz, G. & Crenshaw, C. (2011). Old media, new media: The graphic novel as bildungsroman. Journal of 
Media Literacy Education, 591), 47-53. 

Scull, T. M., & Kupersmidt, J. B. (2010). An evaluation of a media literacy program training workshop for late 
elementary school teachers. Journal of Media Literacy Education, 2(3), 199-208. 

Shaikh, A. N., & Macaulay, L. (2001). Integrating groupware technology into the learning environment. Research in 
Learning Technology, 9(2), 47-63. 

Simonds, T. A., & Brock, B. L. (2014). Relationship between age, experience, and student preference for types of 
learning activities in online courses. The Journal of Educators Online, 11(1), 1-19. 

Slaouti, D. (2001). Student ‘voices’ and course development: Reading the signs on a distance course in educational 
technology. Research in Learning Technology, 9, 62-72. 

Sosa, T. (2009). Visual literacy: the missing piece of your technology integration course. TechTrends: Linking 
Research and Practice to Improve Learning, 55(2), 55-58. 

Stanley, O. L. (2006). A comparison of learning outcomes by “in-course” evaluation techniques for an on-line 
course in a controlled environment. The Journal of Educators Online, 5(2), 1-16. 

Stewart-McKoy, M. A. (2014). “Digitize me”: Generating e-leaming profiles for media and communication students 
in a Jamaican tertiary-level institution. The Journal of Educators Online, 11(1), 1-29. 

Stokrocki, M. (2014). Art treasure quest in Second Life: A multi-literacy adventure. Art Education, 67(2), 37-42. 

Sur, E., Unal, E., & Iseri, K. (2014). Primary school second grade teachers’ and students’ opinions on media 
literacy. Comunicar, 21(42), 119-127. 

Sutton, R. (2014). Unlearning the past: New foundations for online student retention. The Journal of Educators 
Online, 11(3), 1-30. 

Trail, M. A., & Hadley, A. (2010). Assessing the integration of information literacy into a hybrid course using 
screencasting. Merlot Journal of Online Learning and Teaching, 6(3), 647- 654. 

Tyler, K., & Hastings, N. B. (2011). Factors influencing virtual patron satisfaction with online library resources and 
services. The Journal of Educators Online, 8(2), 1-34. 

Valerie, L., & Abed, F. (2013). Comic life + writing = motivated student writers: Incorporating visual graphics to 
teach writing. Preparing Teachers to Teach Writing Using Technology, 163-177. 

Wahidin, K. O., & Meerah, S. M. (2013). Concept mapping in chemistry lessons: Tools for inculcating thinking 
skills in chemistry learning. Journal of Baltic Science Education, 12(5), 666-681 . 

Wallace, P. (2011). Student-community collaboration to construct mobile learning games. Educause Quarterly. 

Wang, L. T., & Lee, K. C. (2014). The study of learners’ preference for visual complexity on small screens of 

mobile computers using neural networks. The Turkish Online Journal of Educational Technology, 13(2), 168- 
180. 

Wang, V. C. X., & Kreysa, P. (2006). Instructional strategies of distance education instructors in China. The Journal 
of Educators Online, 5(1), 1-25. 

Wheeler, S. (2007). The influence of communication technologies and approaches to study on transactional distance 
in blended learning. Research in Learning Technology, 15(2), 103-117. 

Wichadee, S. (2014). Students’ learning behavior, motivation and critical thinking in learning management systems. 
The Journal of Educators Online, 11(3), 1-21. 

Willis, L.R., & Lockee, B. B. (2009). Examining the use of colored overlays with field dependent, reading disabled 
children. Journal of Visual Literacy, 27(2), 155-178. 

Xu, Y. J., Meyer, K. A., & Morgan, D. (2008). Piloting a blended approach to teaching statistics in a college of 
education: Lessons learned. The Journal of Educators Online, 5(2), 1-20. 

Yeh, H. T., & Lohr, L. (2010). Towards evidence of visual literacy: Assessing pre-service teachers’ perceptions of 
instmctional visuals. Journal of Visual Literacy, 29(2), 183-197. 

Young, J. S., & Dannie, R. (2012). Linking learning: Connecting traditional and media literacies in the 2P‘ century 
learning. Journal of Media Literacy Education, 4(\), 70-81. 


47 



Examining the Role of Emotion in Public Health Education Using Multimedia 

Sungwon Chung 

Texas Tech University 
College of Education 
BOX 41071, Texas Tech University 
Lubbock, TX, U.S.A. 

Kwangwoo Lee 

Daegu University 

College of Economics and Business Administration 

201, Daegudae-ro 
Gyeongsangbuk-do, South Korea 

Jongpil Cheon 

Texas Tech University 
College of Education 
BOX 41071, Texas Tech University 
Lubbock, TX, U.S.A. 

Descriptors: emotion, multimedia learning, motivated cognition, health education 

Abstract 

This completed study investigates how learners’ emotions, induced by video clips in different modality 
conditions (i.e., written and spoken), influence learning performance and mental effort during test in a multimedia 
learning environment. A total of 90 students watched video clips with various emotional tones (i.e., positive, 
negative, and coactive) and took recall, retention tests, and rated their mental effort for each test. The results of this 
study revealed that only in the written-text condition, positive groups outperformed negative groups on retention test 
scores; however, they did not differ from coactive groups. Recall test scores and mental effort for the test did not 
differ among the three emotional groups. Implication, limitation and directions for future research are discussed. 

Introduction 

Learners’ emotion can be induced by various audio and/or visual multimedia elements with emotional tone 
which include on-screen texts, narrations, photographs, animations, and videos (Um, Plass, Hayward, & Homer, 

201 1). Emotion is associated with motivation and cognition (learning) (Goetz, Frenzel, Pekrun, Hall, & Ludtke, 
2007; Pekrun & Stephens, 2010; Um et al., 2011). Previous studies have argued that positive emotion improves 
learning by enhancing attention, motivation for learning, and cognitive performance (e.g., test scores), while 
negative emotion could impede the learning processes. 

However, they have not considered the intensity (i.e., arousal levels) of emotional experience. According to 
emotion theories (e.g., Lang, 2006; Pekrun, Goetz, Frenzel, Barchfeld, & Perry, 2011; Russell, 2003), emotion is 
composed of two dimensions, valence (positive/negative) and arousal (calm/arousing). Therefore, the effects of 
positive emotion may be different between calm (e.g., relaxed) and arousing (e.g., enjoyment) levels. Some types of 
negative emotions (e.g., stress and anxiety) may be effective for learning if they facilitate learners’ motivation to 
avoid academic failures (e.g., Pekrun, Elliot, & Maier, 2009). However, there is little research investigating the 
effects of valence in multiple levels of arousal in educational settings. 

This study is grounded by the Limited Capacity Model of Motivated Mediated Message Processing 
(LC4MP: Lang, 2006) guiding to examine how valence and arousal interactively influence motivated cognitive 
processing of mediated content. According to the LC4MP, in a moderately arousing level, memory performance is 
suggested to be greater with negative emotion than positive emotion (see Figure 1). Thus, the current study focuses 
on the effects of valence in a moderately arousing level. In addition, the LC4MP suggests that people can feel mixed 
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emotional feelings (i.e., coaetive) while proeessing both pleasant and unpleasant information simultaneously (i.e., 
when media users learn both information about benefits and side effeets of a medieine). Therefore, the current study 
has the following research questions: 

(a) Does the different type of emotions (positive, negative, and coactive) differently influence students’ 
learning performance (retention and recall test scores) in multimedia learning? 

(b) Does the different type of emotions (positive, negative, and coactive) differently influence mental effort for 
each test? 

(c) Do the effects of emotions differ in different modality conditions (i.e., written and spoken)? 

Literature Review 

LC4MP 

Human emotion, motivation, and cognitive systems are intertwined and influence each other (Cacioppo & 
Gardner, 1999; Lang, 2006). According to the LC4MP (Lang, 2006), human has two underlying motivational 
systems, called appetitive and aversive systems. The appetitive system activates in response to pleasant stimuli, 
eliciting positive emotion. The aversive system activates in response to unpleasant stimuli, eliciting negative 
emotion. The intensity of pleasantness or unpleasantness determines arousal levels of positive or negative emotion. 
While eliciting emotional responses, as shown in Figure 1, the appetitive and aversive systems differently regulate 
the amount of resources allocated to cognitively processing information (Lang, 2006). 



Arousal of learner emotions 

Figure 1 . The Relationship Among Emotion, Motivation, and Cognition 

A human has the instinct for approaching pleasant but avoiding unpleasant stimuli (Cacioppo & Gardner, 
1999; Lang, 2006). In a safe environment (with little threat), the human instinctively wants to seek and remember as 
much pleasantness as possible. Thus, in calm levels of arousal, memory capacity is greater for appetitive than 
aversive activation. On the other hand, the aversive system functions to identify and remember as much 
unpleasantness as possible. In a greater threatening environment, the aversive system more agilely activates to 
protect individuals from any danger. In arousing levels, thus, memory capacity is greater for aversive than appetitive 
activation. Overall, greater motivational activation leads to better memory capacity. If a life-threatening stimulus 
(e.g., death) is presented, the aversive activation turns down to remember the repulsive unpleasantness and rather 
switches for internally retrieving escaping information. 

In addition, the LC4MP suggests that the appetitive and aversive systems independently work. Thus, 
human can experience simultaneous feelings of positive and negative emotions (e.g., thinking about side effects and 
benefits of a medicine). The simultaneous activation of the appetitive and aversive systems is referred as 
coactivation. According to the LC4MP (e.g., Lang 2006; Norris, Bailey, Bolls, & Wise, 2012), memory capacity 
with coactivation is greater in calm levels than aversive activation only, and greater in arousing levels than 
appetitive activation only. However, because of the inherent complexity of coactive infonuation, cognitive overload 
is more likely to occur during coactivation. 
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Emotion and Multimedia Learning 

Emotion studies in education have suggested the utilization of positive emotion in instructional designs 
using multimedia. For example, positive emotion can be induced by such as aesthetic design elements, bright warm 
colors, human-like shapes (Um et ah, 2011), pictorial illustrations (Park & Lim, 2007), video-based instructions 
(Chen & Sun, 2012), and background music (Fassbender, Richards, Bilgin, Thompson, & Fleiden, 2012). They have 
demonstrated the benefits of positive emotion in enhancing motivation, satisfaction, behavioral intentions, cognitive 
interest, cognitive perceptions, and knowledge acquisition and retention. Flowever, the findings were addressed with 
positive emotion without the consideration of its arousal level (e.g., Um et ah, 2011). Moreover, there has been no 
research examining the effects of coactive emotion, although learners could process learning materials which may 
cause the mixed feelings. 

Further, multimedia learning materials can be presented in a different modality format (i.e., written or 
spoken text). For example, the presentation of information can be displayed as on-screen text or delivered as 
narration only. Since human has a separate channel of visual and auditory processing, each modal’s cognitive 
capacity is also different and the time of holding the information may also differ (Mayer & Moreno, 2003). Thus, we 
aims to examine how learners’ emotions (i.e., positive, negative, and coactive) affect their learning in different 
modality conditions. 


Method 


Data Collection 

Participants in this study were 90 undergraduate students (« = 45 for a written text condition; « = 45 for a 
spoken text condition) from a large southwestern university in the United States. This study used an experimental 
design with 3 levels of valence (positive, negative, and coactive) and 2 types of modality (written and spoken). 

For the experiment, two video clips with positive tone (i.e., reversing diabetes) and two video clips with 
negative tone (i.e., suffering diabetes) were created. All video clips have the same running time (2’ 35”). In regards 
with valence conditions, positive, negative, and coactive groups watched two positive, two negative, or one positive 
+ one negative video clips, respectively. In regards with modality conditions, the written-text condition contained 
captions within the video clips. The spoken-text condition contained narrations, instead of the captions, in the video 
clips. 

Dependent variables included (a) retention test, (b) recall test, (c) mental effort for retention test, and (d) 
mental effort for recall test. 

Learning content for this study is presented after the video clips. Total 12 major risk factors causing Type 2 
diabetes were instructed. Thus, recall test scores could be up to 12 points; retention test scores could be also up to 12 
points; and mental efforts were measured using a one-item 9-point scale (1 = extremely low; 9 = extremely high). 
This study followed three phases in the following order: 

• emotional video clip (treatment) with learning content 

• self-ratings of valence and arousal (for manipulation check) 

• assessment (recall and retention tests) and mental effort for each test 

Manipulation Check 

Participants were asked to indicate how positive, negative, and arousing they felt about the video clips they 
just watched on a three 9-point Likert scale (e.g., 1= not at all positive; 9 = extremely positive) (Norris et ah, 2012). 
ANOVA analysis indicated significant differences in positive valence among positive, negative, and coactive video 
clips, 7^(2, 87) = 54.427,/? < .001. A post-hoc test revealed that positive valence was greatest for positive video clips 
(M= 6.67; SD = 1.73), followed in order by coactive (M= 5.33; 5D = 1.95) and negative video clips (M= 2.20; SD 
= 1.38). ANOVA analysis also found significant differences in negative valence among the three types of video 
clips, F{2, 87) = 38.618, p < .001. A post-hoc test revealed that negative valence was greatest for negative video 
clips (M= 6.40; SD = 2.14), followed in order by coactive (M = 4.60; 5D = 1.81) and positive video clips (M= 2.27; 
5D= 1.46). 


Results 

ANOVA analysis showed no interaction effect between valence and modality on retention test scores {F{2, 
84) = 2.779, p = .068), on recall test scores (F’(2, 84) = .942, p = .394), on mental effort for retention test {F{2, 84) 

= 332, p = .718), and on mental effort for recall test {F{2, 84) = .487,/? = .616). Additional analyses were performed 
to examine the effects of positive, negative, and coactive emotions in each modality condition (see Table 1). 
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Regarding the written-text condition, ANOVA analysis revealed significant differences on retention test 
scores, F{2, 42) = 3.507,/? < .05. A post-hoc test indicated that positive emotion 8.20, SD = 2.34) 
outperformed negative emotion 6.13; SD = 2.03),/? < .05; however, coactive emotion 7.13, SD = 2.03) 
did not differ from positive (p = .367) and negative emotion (p = .413). 

However, ANOVA analyses found no significant difference on recall test scores {F{2, 42) = 2.145,/? 

= .130), on mental effort for retention test (7^(2, 42) = .195,/? = .823), and on mental effort for recall test {F{2, 42) 

= .170,/? = .844). 

Regarding the spoken-text condition, ANOVA analyses found no significant difference on retention test 
scores (7^(2, 42) = .381,/? = .686), on recall test scores (7^(2, 42) = .161,/? = .851), on mental effort for retention test 
(7^(2, 42) = .457,/? = .636), and on mental effort for recall test scores (7^(2, 42) = 1.136,/? = .331). 


Table 1. Mean and standard deviation 



Written-text condition (N = 45) 

Spoken-text condition (N = 45) 

Retention test scores 

Positive 

Negative 

Coactive 

Positive 

Negative 

Coactive 


M=8.20 
SD = 2.34 

M=6.13 
SD = 2.03 

M=7.13 
SD = 2.03 

SD = 2.58 

M=7.67 
579= 1.99 

M^7.73 
579 = 2.31 

Recall test scores 

Positive 

Negative 

Coactive 

Positive 

Negative 

Coactive 


M=5.93 
SD = 2.49 

M=4.33 
SD = 2.32 

M=5.73 
SD = 2.09 

M=5.40 
SD = 2.47 

M=5.07 
579 = 2.05 

M=4.93 
579 = 2.40 

Mental effort for Retention test 

Positive 

Negative 

Coactive 

Positive 

Negative 

Coactive 


M=6.87 
5D= 1.41 

M=6.60 
5D= 1.50 

M=6.93 
5D= 1.71 

M=5A1 
579= 1.64 

M=5.B,7 
579 = 2.13 

M=6.13 
579= 1.96 

Mental effort for recall test 

Positive 

Negative 

Coactive 

Positive 

Negative 

Coactive 


M=6.00 
SD = .926 

M=5.93 
5D= 1.53 

M=6.20 
579= 1.37 

M=5.80 
579= 1.47 

M=5.00 
579 = 2.00 

M=5.93 
579= 1.98 


Discussion 

This study was a first trial to examine how positive, negative, and coactive emotions influence learning in a 
multimedia learning environment. The results revealed no interaction effect between valence and modality on 
learning performance (retention and recall test scores) and mental effort for each test. However, in each modality 
condition, the effects of learners’ emotions were different. Specifically, retention test scores were different only in 
the written-text condition; however, they did not differ in the spoken-text condition. We conjecture that the emotion 
effect may keep longer sustained in the written-text condition because visual processing (i.e., reading) could require 
more cognitive effort than auditory processing. Moreover, the information of the spoken text tends to be transient 
and processed relatively more quickly. Specifically, the results showed that positive emotion outperformed than 
negative emotion in the written-text condition. Positive emotion may be sustained longer than negative emotion; 
whereas negative emotion seems to be quickly diluted by additionally provided learning content on the risk factors 
of Type 2 diabetes. 

Moreover, although retention test scores were different among the three types of emotions, recall test scores 
did not differ. The results are possible because recall requires more cognitive effort and deeper learning process. In 
other words, although recognition of useful information could be successful, it does not ensure successful recall of 
the information. We assume that emotions could help learners, without effort, motivated with the given content but 
deeper learning should involve learners’ active cognitive effort and processing. 

Furthermore, mental effort was not different among the three types of learners’ emotional states. Mental 
effort indexes learners’ conscious effort to think about what they learned. Thus, the findings are consistent with the 
notion that emotions have a motivational power which automatically allocates cognitive resources to the processing 
of the learning content. 

This study has many limitations due to the nature of experimental research, and thus addresses several 
directions for future research. First of all, this study did not examine the effects of emotional tones in different 
arousal levels. As shown in Figure 1, depending on the arousal levels, the effects of three types of emotions may be 
counteracted. Further, future research needs to explore how different the sustaining time of the different types of 
emotions is and how they differently influence learning. The effects of coactive emotion were not found in this study. 
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although coactive emotion exists and successfully manipulated in this study. This study was conducted in a specific 
learning context and media context. So, future research may explore the effects of coactive emotion in different 
mediated learning contexts. 
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Abstract: This survey research investigates online facilitation strategies in collectivist cultures 
within the “Community of Inquiry” framework. Four hundred and ninety-five students registered 
in a Southern European online university participated in this study. Preliminary findings show that 
online students in collectivist cultures value reflection within the online learning community, long 
for affective expression and group cohesion, value collaborative activities and are able to self- 
organize to increase presence in online programs. 

Background 

Online education is on the rise in higher education. The Online Learning Consortium reports (Allen & 
Seaman, 2013) on online education in the United States that: (1) the number of students taking at least one online 
course is 7.1 million, which represents 33% of all higher education students; and (2) sixty-six percent of academic 
higher education institutions identify online learning as a critical part of their long-term strategy. While this scenario 
offers challenges to higher education, it also offers opportunities to widen access to education, enhance the quality 
of teaching, and reduce the cost of higher education. 

In the study of online education within higher education, much has been discussed about the value in 
creating communities of inquiry in general, and in online environments in particular. Communities of inquiry are 
“where interaction and reflection are sustained; where ideas can be explored and critiqued; and where the process of 
critical inquiry can be scaffolded and modeled. . . [a] community of inquiry must include various combinations of 
interaction among content, teachers, and students” (Garrison & Cleveland-Innes, 2005, p. 134). It is also argued that 
online students should attain high levels of critical thinking and knowledge construction as result of well-designed 
online learning experiences (Aviv et ah, 2003; Wu & Fliltz, 2004). Online learning experiences reach their potential 
when they aim to support knowledge construction as a process of creation and innovation developed collaboratively. 

With the growth of Web 2.0 technologies (e.g., social networking sites, blogs, wikis, and video sharing 
sites), consumers of information transform into producers of infomiation. This trend is invading the educational 
arena (Downes, 2006), expanding the role of the learner, and shifting the role of instructor. Members of an online 
learning community rely increasingly on trust and reciprocity to support their own and one other’s learning. Such 
evolution in online education leads to alternative ways to facilitate online learning. 

According to Goodyear, Salmon & Steeples (2001), facilitators need to perform a plethora of roles ranging 
from process facilitator (facilitating the range of online activities that are supportive of student learning) and content 
facilitator (concerned directly with facilitating the learners’ growing understanding of course content) to 
adviser/counselor (working on an individual/private basis, offering advice or counseling learners to help them get 
the most out of their engagement in a course) and researcher (concerned with engagement in production of new 
knowledge of relevance to the field). 

From a social constructivist perspective, it is expected that the facilitator be an active promoter of the 
learning community as well as a content expert. The sustainability of an online learning environment is a direct 
result of an increase in reflection within the community, which in turn increases the levels of participation and trust 
among its members (Salmon, 2000). Online discussions create opportunities for students to construct meanings 
together and integrate new knowledge into their prior experiences (Rourke & Anderson, 2002). Online discussions 
can serve as a platform for students and instructors to interact in a social environment without boundaries of time 
and distance, promoting students’ critical thinking and helping students reflect on their ideas (Brooks & Jeong, 

2006; Hew & Cheung, 2008; Wang, 2008). 

Research has identified several problems related to online discussions, such as limited student participation 
(Hewitt, 2005), inadequate critical analysis of peers’ ideas (Rourke & Anderson, 2002), lack of motivation, 
commitment, and time, and failure to communicate effectively (Brooks & Jeong, 2006). To address some of these 
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pitfalls, a number of facilitation strategies, mostly focusing on the instructor as facilitator or moderator, have been 
described in the literature (Anderson et ah, 2001). Although tutors and instructors play a critical role in online 
discussion environments, their domination may result in an instructor-centered discussion, suppressing students’ 
active participation (Rovai, 2007). For example, instructors may not be able to fulfdl all the facilitation 
responsibilities because of the high time commitment required (Rourke & Anderson, 2002). Managing a large 
discussion group online may be overwhelming. 

Although instructor-led discussions do not necessarily result in instructor-dominated discussion, having the 
instructor at the center of the discussion may create an “authoritarian presence” (Rourke & Anderson, 2002, p. 4) not 
conducive to genuine conversations. While in Garrison, Anderson, and Archer’s (2000) Community of Inquiry 
framework, the authors give most of the moderation activities to teachers, they acknowledge that teaching presence 
can also be achieved through meaningful interaction among students. In this line of reasoning, facilitation is a shared 
responsibility among instructors and students, changing the traditional role of the instructor from having much of the 
control in the instmctional relationship to becoming more of a participant and fellow learner. Baran and Correia 
(2009) examined student-led facilitation strategies as a way to overcome the challenges of instructor-dominated 
facilitation, enhance the sense of learning community, and encourage student participation in online discussions. In 
this naturalistic study, three facilitation strategies - inspirational, practice-oriented, and highly-structured - were 
identified 


Research Purpose 

Even though much research in online facilitation has been conducted, most of it has occurred in North 
American contexts of practice, which are defined by Flofstede (2001) as individualist cultures. Flofstede’s extensive 
work on comparing values, institutions, and organizations across nations has established a four-dimensional model 
of differences among national cultures. The four dimensions are: 

1) Power Distance - describes the extent to which the less powerful members of institutions and organizations 
within a nation expect and accept that power is distributed unequally; 

2) Individualism / Collectivism - describes the ties between individual members of a society. In individualist 
societies, the ties between individuals are loose, and everyone is expected to look after himself or herself 
and his or her immediate family; in collectivist societies, people from birth onwards are integrated into 
strong, cohesive ingroups, which throughout people's lifetimes continue to protect them in exchange for 
unquestioning loyalty; 

3) Masculinity / Femininity - describes the degree to which dominant values of a society are distinctively 
“masculine” or “feminine,” and the degree to which social gender roles are distinct or overlapping; 

4) Uncertainty Avoidance - describes the extent to which the members of a culture feel threatened by 
uncertain or unknown situations and try to avoid such situations. 

The Individualism / Collectivism dimension, which is particularly evident between collectivist cultures 
(e.g., Portugal, Brazil, Chile, and Costa Rica) and individualist cultures (e.g., USA, Canada, and other English- 
speaking countries) is addressed in this study. Such differences within the Individualism / Collectivism dimension 
impact teaching and learning (Flofstede, 1986). According to this author. Individualism / Collectivism describes the 
ties between individual members of a society. In individualist societies, the ties between individuals are loose, and 
everyone is expected to look after himself or herself and his or her immediate family; in collectivist societies, people 
from birth onwards are integrated into strong, cohesive ingroups, which throughout people's lifetimes continue to 
protect them in exchange for unquestioning loyalty. 

This research study aimed at investigating online facilitation strategies in collectivist cultures within the 
Community of Inquiry framework (Garrison, Anderson & Archer, 2000). The following questions guided the 
research: 

1 . To what extent are online students engaged in online learning (levels of teaching, social and cognitive 
presence)? 

2. Flow often do students engage in online collaborative activities (e.g., online discussions or online teamwork 
- two or more peers)? What types of collaborative activities are used in class? Which role(s) do students 
perform in these activities? 
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Online facilitation is identified here as one of the most important roles of the online instructor. By 
facilitating the online experience, the instructor helps to bring about learning outcomes through unobtmsive 
assistance, guidance, and monitoring. 

The Community of Inquiry offered a framework to analyze online learning experiences through the development of 
three interdependent components, as follows: 

• Social presence- “the ability of participants to identify with the community (e.g., course of study), 
communicate purposefully in a trusting environment, and develop inter-personal relationships by way 
of projecting their individual personalities” (Garrison, 2009, p. 353); 

• Teaching presence- “the design, facilitation, and direction of cognitive and social processes for the 
purpose of realizing personally meaningful and educationally worthwhile learning outcomes” 
(Anderson, Rourke, Garrison & Archer, 2001, p. 6); 

• Cognitive presence- “the extent to which learners are able to construct and confirm meaning through 
sustained reflection and discourse” (Garrison, Anderson & Archer, 2001, p. 9). 

Methods 

A survey research approach was used, which is “aimed primarily at tapping the subjective feelings of the 
public” (Fowler, 2013, p. 2). The study took place in a Southern Europe online university, refereed here as Aurora 
University (AU). AU offers education to more than 12,000 students in 33 countries over five continents, including 
migrant communities and Portuguese-speaking countries that are home to over 240 million people across the globe 
(e.g., Brazil, Mozambique, Angola, Cape Verde, and Guinea-Bissau). 

The Community of Inquiry survey has been extensively tested (Arbaugh et ah, 2008) and validated (Swan 
et ah, 2008). Swan at al. report that Cronbach’s Alpha equals to 0.94 for Teaching Presence, 0.91 for Social 
Presence, and 0.95 for Cognitive Presence. The Community of Inquiry survey consists of 34 Likert scale -type items 
(1-Strongly Disagree to 5-Strongly Agree). The items are organized under three categories: teaching, social and 
cognitive presence. A translated version into Portuguese of the Community of Inquiry survey was used as data 
collection instrament. A section on demographics and one open-ended question about online collaborative activities 
were added to the survey. The survey was administered online to AU students over 2 1 years old who were asked to 
complete the survey in reference to the online course(s) in which they were enrolled. Previously to this full-blown 
study, a translated version of the Community of Inquiry survey was filled out by a group of 30 students and yielded 
a Cronbach’s Alpha of 0.885. 


Analyses and Resnlts 

Out of 495 respondents, 53% were female and 47% male with ages ranging from 22 to over 65 years old. 
Forty-one percent of the participants were between 35 and 44 years old. The majority of the respondents (90%) lived 
in Portugal. Sixteen percent were unemployed at the time of the survey and pursuing a degree at AU as an 
opportunity to advance their knowledge and skills, shift careers and/or finding a job. Sixty-six percent of the 
respondents were pursuing a Bachelor's degree while 23% were graduate students. A small group of 55 students 
were attending courses in the Lifelong Learning program. Ninety percent of the students were experienced online 
learners who have taken more than three courses at AU. 

Since levels of social, teaching and cognitive presence determine to what extent online students are 
engaged in an educational experience the data analysis focused initially on determining these levels. For example, 
high levels of these three interdependent dimensions indicate cognitive and social development as well as the 
facilitation of meaningful learning. In regards, to the extent online students were engaged in online learning the 
following results were obtained (Table 1). The survey items followed the categorization proposed by Swan et al. 
(2008). Examples are. Design & Organization, Facilitation, Direct Instruction, Affective Expression, Open 
Communication, Group Cohesion, Triggering Event, Exploration, Integration and Resolution. 


55 



Table 1 - Highest/lowest percentages on the three types of online presence (social, teaching and cognitive). For each 
type of presence the two highest (in italics) and the two lowest combined percentage of agreement (Strongly Agree 
+Agree) are reported. 


Social presence Strongly Agree + 
Agree = 79.59% 
Strongly Agree + 
Agree = 76.22% 


1 felt comfortable interacting with other course participants 
(Open Communication) 

I felt comfortable conversing through the online medium 
(Open Communication) 


Strongly Agree + 
Agree = 69.87% 
Strongly Agree + 
Agree = 67.15% 


I was able to form distinct impressions of some course 
participants (Affective Expression) 

I felt that my point of view was acknowledged by other 
course participants (Group Cohesion) 


Teaching 

Strongly Agree + 

The instructor provided clear instructions on how to 

presence 

Agree = 92.53% 

participate in course learning activities. 


Strongly Agree + 

The instructor clearly communicated important due 


Agree = 91.38% 

dates/time frames for learning activities. 
(Design & Organization) 


Strongly Agree + 

The instractor provided feedback that helped me understand 


Agree = 70.47% 

my strengths and weaknesses. 


Strongly Agree + 

The instructor provided feedback in a timely fashion. 


Agree = 69.86% 

(Direct Instruction) 


Cognitive 

presence 


Strongly Agree + 
Agree = 91.96% 
Strongly Agree + 
Agree = 90.33% 

Strongly Agree + 
Agree = 78.79% 
Strongly Agree + 
Agree = 74.95% 


/ utilized a variety of information sources to explore 
problems posed in this course (Exploration) 

Reflection on course content and discussions helped me 
understand fundamental concepts in this class 
(Integration) 

I have developed solutions to course problems that can be 
applied in practice (Resolution) 

Online discussions were valuable in helping me appreciate 
different perspectives (Exploration) 


Social presence exhibits lower levels of agreement compared to teaching or cognitive presence. In terms of 
social presence, open communication (e.g., feeling comfortable when interacting and conversing with other online 
students) shows the highest percentage of agreement, but affective expression (e.g., forming distinct impressions 
about the course participants) and group cohesion (e.g., feeling their point of view acknowledged) obtained the 
lowest percentages. Teaching presence displays high agreement percentages in design & organization (e.g., clear 
instructions on how to participate in course activities; and, communication of important due dates/time frames for 
learning activities) and lowest in regards to direct instruction (e.g., feedback provided and timeliness of the 
feedback). 

Finally, cognitive presence shows the highest percentage of agreement in relation to exploration (e.g., use 
of a variety of information sources to explore problems), but also the lowest one (e.g., online discussion and its 
value on helping students to appreciate different perspectives). This seems to evidence some contradiction within the 
Exploration category. Cognitive presence also shows the highest percentage of agreement in integration (e.g., value 
of reflection on course content and discussions), but the lowest percentage in resolution (e.g., course problem and its 
application in practice). Overall the percentage of agreement are high for these three types of online presence and it 
may be argued that this translate into a significant level of student engagement in online learning. 

Sixty-three percent of the students reported being engaged in online collaborative activities ranging from 
program orientation to team projects and field work. Most of the students were part of self-organized groups to 
address questions and issues raised in their online classes and to serve as support group. Table 2 shows the types of 
online collaborative activities students engaged in and respective frequencies. 
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Table 2 -Types of online collaborative activities and frequencies 


Online collaborative activities 

% 

Self-organized online groups led by the students to address questions / issues raised in 
their online classes and to serve as support group (synchronous and asynchronous 
communication) 

40.2% 

Online discussions moderated by the instmctor to address course content (synchronous 
and asynchronous communication) 

23.0% 

Team projects and accompanying reports 

13.5% 

Class presentations (real-time and pre-recorded) 

7.6% 

Open-fomms create and moderated by the course instmctor to discuss course-related 
questions and issues 

6.1% 

Team projects (including presentations to the class) 

4.5% 

Field trips and accompanying reports 

3.0% 

Program orientation (for new students) 

2.3% 


Conclusions 

Preliminary findings of this study bring some understanding about online learning within the Community 
of Inquiry framework in collectivist cultures. For example, a high percentage of students are engaged in online 
collaborative activities, occurring most of them in self-organized online groups led by the students. These groups do 
not only address questions and issues raised in their online programs, but as importantly they act as support groups. 
They might have been spurred out of the students’ need for a higher level of social presence in the online courses. 

The purpose of this study is not to look at online learning in collectivist cultures and compare it to 
individualist cultures. By the contrary, the contribution of this study is to offer a look into a phenomenon that has 
not been extensively researched since much of the research in online facilitation has occurred in North American 
contexts of practice, where individually constructed knowledge is more valued. Initial findings provide evidence that 
online students in collectivist cultures value reflection within the online learning community, long for affective 
expression and group cohesion, value collaborative activities and are able to self-organize to increase presence in 
online programs. 
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Abstract 

The purpose of this study was to examine the internship activities of practicing graduate student library 
science interns. To be prepared to teach critical twenty-first century skills to students, future school librarians need 
in-depth understanding of the standards guiding education in public schools. Standards help define common ground 
in terms of sets of skills and dispositions, for use in preparing students to function as learners in an increasingly 
digital environment (Mardis & Dickinson, 2009). 

This paper will provide an analysis of survey responses and interviews from 114 library science interns 
regarding their professional internship experiences with specific sets of standards. The standards under review for 
this study are American Association of School Librarians (AASL) Standards for Initial Preparation of School 
Librarians and International Society for Technology in Education (ISTE) Standards for Coaches. 

Introduction 

The role of the school librarian has changed in recent years due in large part to emerging technologies. 
Studies have proven the value of school librarians in contributing to student success. Because education today 
involves dealing with rapidly shifting technologies the librarian role must be fully integrated into the instructional 
practice if librarians are to effectively serve. To be fully integrated, school librarians must have an understanding of 
professional standards in order to meet the needs of those they will serve. Therefore, to improve collaborative efforts 
and meet instructional needs graduate school educators for future librarians realize it is necessary to make the 
teaching of professional standards a top priority. 

Librarians must look beyond the individual role and plan, to the broader, more expansive national and 
international view, which demands attention to state, national and international standards. Sets of standards have 
been tested and vetted by informed educators to determine if and how they may provide support and direction for 
both classroom teachers and librarians (Church, Dickinson, Everhart, & Howard, 2012). These standards help define 
common ground in terms of sets of skills and dispositions, for use in preparing students to function as learners in an 
increasingly digital environment (Mardis & Dickinson, 2009). 

This study examines the standards practiced during library and information science internship courses and 
investigates the kinds of experiences interns in library and information science graduate programs report they have 
experienced. 


Need for Study 

As the positions of school librarians come under increasing scrutiny, it becomes ever more important to be 
able to provide support for the classroom teacher. In doing so librarians are able to further show their contributions 
to schools. Educators are increasingly bombarded with new technology and school librarians who are prepared to 
provide teachers with the support they need will make it a priority to keep up with advances and changes in 
technology and will also be keenly aware of the national standards as they relate to and support teaching with 
technology (Choi & Rasmussen, 2006; Ekhaml & Hubbard, 2003). Efforts focused on technology skills and the 
integration of technology in the classroom are critical. Having a solid foundation in the use and integration of 
standards in the instructional process is crucial to the well-prepared librarian. With this background beginning 
librarians will feel empowered (Hughes-Hassell & Hanson-Baldauf, 2008), be able to guide classroom teachers and 
help determine the best uses of technology for the best outcome. (Ballew, 2014). School librarians who are 
particularly well versed in both American Association of School Librarians (AASL) Standards and International 
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Society for Technology in Education (ISTE) Standards for Coaches, as well as being skilled in technology use and 
integration, will be better able to better foster an atmosphere of shared support, respect and rapport with teachers. 

Therefore researchers for this study were interested in the standards practiced during library and 
information science internship courses and the kinds of experiences interns in library and information science 
graduate programs have. The research questions that guided this study included questions and areas of interest to the 
researchers regarding the kinds of clinical experiences in technology integration pre-service librarians perceived 
they participated in within the internship; which critical standards were emphasized within the internship; and any 
feedback on preparing students for leadership roles in school librarianship that could emerge from an examination of 
the data. To answer these concerns and determine practices, processes and results of professional development, 
researchers designed and conducted a web administered survey. 

Methods 

Based upon a review of the empirical research on professional standards, a survey was designed to gather 
self-reported information from library science graduate students. Researchers employed a survey method. This study 
used a 52-item web-based survey, “Final Perceptions of Standards Practiced within the Internship” which was 
developed by the author/researcher and approved by the university’s Institutional Review Board. The survey was 
distributed to library science interns who were drawn from eighteen sections of the professional internship course at 
a southeastern university over a period of time from 2010-2014. The study defined “intern” as a student in an 
extended field experience under the direct supervision of a certified school librarian who is in collaboration with a 
graduate program. Information collected in the survey included year of the internship, gender, and school status, 
and 40 questions asking about program standards and perceptions of technology integration within the internship 
experience. A total of 1 14 interns responded to the study. 

The survey was formatted and disseminated using a web-based survey instrument, Qualtrics. To emphasize 
reasons for the study, introductory information was sent electronically to the selected population along with the 
survey links. A Likert scale was utilized to obtain quantitative data, allowing for self-reporting with individuals 
addressing each question. Open-ended questions allowed respondents to provide more detailed and individualized 
information. Upon culmination of the survey, research software the Statistical Package for Social Science (SPSS), 
was employed, in conjunction with Qualtrics, to facilitate a detailed analysis of the reported data. 

Participants 

Participants interned in different settings as library interns, with 46% at rural settings, 35% at suburban 
settings, and 20% urban settings. 


Table 1 



Response 

% 

Rural 


51 

46% 

Suburban 


39 

35% 

1 

Urban 

1 

22 

20% 


School level participation included 60 % at primary level, 21% middle level, and 19% at secondary level. 


Table 2 


1 Answer 

Response 

% 

Primary 


65 

60% 

Middle 


23 

21% 

Secondary 


21 

19% 


Gender was reported as predominantly female with 9% male and 91% female. The average age of 
respondents was 40.5, with the youngest being 24 and the oldest 59. 

The years in which the respondents reported serving internships were reported as follows: 
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Table 3 


lAnswer 

Response 

% 

2009 


5 

5% 

2010 


15 

14% 

2011 


17 

16% 

2012 


12 

11% 

2013 


40 

38% 

2014 


16 

15% 


Foundation of the Survey 

AASL Standards for Initial Preparation of School Librarians and International Society for Technology in 
Education Standards for Coaches emphasize problem-solving, critical thinking, and analytical skills while giving 
teachers some flexibility and judgment in delivering the curriculum. Specifically AASL Standards for Initial 
Preparation of School Librarians are written for LIS educators to prepare graduates to develop and manage library 
services in school programs (AASL, 2010), while ISTE Standards for Coaches concentrates upon the standards for 
evaluating the skills and knowledge that education coaches need to guide and support teachers in an increasingly 
connected and global society (ISTE, 2012). Both sets focus on acquiring the skills to pursue knowledge. Together 
these standards provide guidance on key skills for instmction, along with logical steps and direction on how to 
deliver instruction in an effective manner to encourage student learning. Both sets of standards also recognize that 
skills and disposition that connect curriculum with real-world application is necessary (Mardis & Perrault, 2008). 
Themes from each set of standards were used for development of survey questions. 

Analysis of Findings 

It is clear from the standards that school librarians must model the vision of a technology aware school and 
be collaborative partners. If students are to make use of and benefit from technology, leaders have to provide 
access, organize, and display and use whatever types of technology are available. It is important to identify 
innovative products for teaching and learning use. We must model technology enhanced experiences in our teacher 
in-service opportunities through first planning technology integrated lessons with teachers and then collaborating 
with teachers to develop inquiry experiences for students. 

The overlap between ITSE Standards for Coaches, in terms of vision and leadership and AASL Standard 4: 
Advocacy and Leadership was evident in our survey results. Respondents reported experiences in connecting, 
sharing, collaborating, and interacting with other educators. The close alignment in these standards, specifically 
AASL 4 and ISTE I focused on demonstrating leadership through networking and building relationships with 
teachers. Working together teachers and librarians can benefit everyone involved. Results indicated experiences 
with these two standards as reported 


ALA/ AASL Standard 

ISTE Standards for 
Technology Coaches 

Activities Practiced and Reported 

AASL Standard 4: Advocacy 
and Leadership 

ISTE Standard 1. Visionary 
Leadership 

• Advocated for learning programs and 
positive learning environments 

• Led collaboration with a variety of 
instructors 

• Participated in professional growth and 
leadership opportunities 

• Advocated for information programs, 
resources, and services 
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Another area where strong evidence was reported with two overlapping standards included the AASL 
Standard 1 and ISTE Standard 2. Both of these standards appeared to be addressed well in the internship: 


ALA/ AASL Standard 

ISTE Standards for 
Technology Coaches 

Activities Practiced and Reported 

AASL Standardl: Teaching for 
Learning 

ISTE Standard 2: Teaching, 
Learning, & Assessments 

• Instructed based on knowledge of diverse 
learning needs and interest of students 

• Collaborated with other educators in the 
integration of 2P‘ Century skills 

• Documented, analyzed, and shared data on 
student achievement 

• Advocated and modeled best practices for 
incorporating technology into instruction 

• Contributed to instruction based on knowledge 
of curriculum standards 


Conclusions 

The information gained through this study may be helpful for programs in deciding which sets of standards 
are most appropriate for a particular program. The results indicate that pre-service librarians are increasingly 
requiring technology competence for effective instructional collaboration. In their dual roles as both school librarian 
and instructional collaborator, mastery of technology and skills as technology coaches are crucial. ISTE Standards 
for Coaches provide a solid foundation from which to prepare future school librarians towards this end. Graduate 
program coursework aligned with AASL Standards for Initial Preparation of School Librarians provide future 
librarians with skills and dispositions to support mastery of technology and associated troubleshooting skills as an 
essential part of their preparation program. While these standards are different, they are not opposing, and together 
they can help to more optimally support instmction. 

Successful graduate programs should identify and concentrate upon standards that enhance their abilities to 
facilitate development of well-prepared, qualified learners who are ready to step into interdisciplinary atmospheres 
that will support a broad range of diversity in faculty, students, and student interests. Without doubt, knowledge of 
the standards and their impact in this digital information era must be our focus as we strive to help learners navigate 
life through deeper, more meaningful, lifelong learning. 
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Abstract 

A posttest-only, quasi-experimental study was conducted to investigate the effect of two problem-based 
learning environments on students’ self-regulation of learning and students’ level of cognitive load. The study 
involved 40 graduate nursing students from two intact nursing courses where problem-based learning was the 
principal method of instruction. Twenty students from an online course and 20 students from a face-to-face course 
received one ill-structured problem per week in their respective courses over the 4-week duration of this study. All 
participants completed the Motivated Strategies for Learning Questionnaire after their 4-week participation in the 
study to describe their learning motivation and their use of learning strategies. Students’ 4-week participation in the 
study also involved a weekly measure of students’ cognitive load after the completion of an ill-structured problem 
distributed weekly. A total of four cognitive load measures (i.e., one measure per week) were generated to assess 
students’ cognitive load across the duration of the study. 

The results of the study revealed that students did not differ in motivation, cognitive load, and eight of nine 
self-regulated learning strategies. A MANOVA statistical test indicated that students who received problem-based 
learning in the online course used fewer metacognitive strategies than students who received problem-based learning 
in the face-to-face course. Students in the online course were comparable to students in the face-to-face course with 
respect to the learning strategies rehearsal, elaboration, organization, critical thinking, peer learning, help seeking, 
effort regulation, and time and study environment. A second MANOVA statistical test indicated that students who 
received problem-based learning in an online course did not differ from students who received problem-based 
learning in the face-to-face course with respect to learning motivation. Students in both problem-based learning 
environments reported comparable intrinsic goal orientation, extrinsic goal orientation, control of learning beliefs, 
task value, test anxiety, and self-efficacy for learning and performance. Both groups of students experienced a high 
level of cognitive load but did not experience different levels of cognitive load, as determined by a mixed ANOVA 
statistical test. 


Introduction 

Nearly 33% of postsecondary students are enrolled in at least one online course where students receive 
80% or greater of their course instruction by Internet (Allen & Seaman, 2013). This percentage is projected to rise as 
high as 50% by the end of 2014, increasing the number of students with a minimum of one online course enrollment 
(Christensen, Horn, Caldera, & Soares, 2011). The projected rise in student enrollment in online courses will likely 
give rise to an expansion of the common practice of replicating traditional, face-to-face course design in online 
courses (Bonk & Dennen, 2003). Vaughn (2010) suggested that such replication occurs when postsecondary 
instructors fail to make the transformational shift from the practice of disseminating information to the practice of 
creating learning environments that enable students to co-construct knowledge. The inability of postsecondary 
instructors to make this transformation has forced many students to contend with poorly designed online courses 
(Bannert, 2004; Zumbach & Mohraz, 2008). 

Several scholars (Amadieu, Gog, Paas, Tricot, & Marine, 2009; Evans & Douglas, 2008; Schnotz & Heiss, 
2009) indicate that poor instructional design contributes to excessive cognitive load in students who receive 
instruction in online courses. Paas and Van Merrienboer (1994) defined cognitive load as the load that performing a 
particular task imposes on the limited resources of students’ working memory. Thus, poor online course design, 
resulting from the practice of transferring traditional face-to-face courses to online courses, may place an excessive 
burden on the limited resources of students’ working memory. 

The transfer of traditional face-to-face course design to online courses not only raises the potential for 
excessive cognitive load but also shows the wide regard for online courses as delivery systems (Allen & Seaman, 
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2013). As such, a number of problem-based learning instructors at the postsecondary level have recently begun to 
bansfer their face-to-face problem-based learning courses to online course environments (Chia-Wen & Yi-Chun, 
2013; Duncan, Smith, & Cook, 2013). This transfer of face-to-face problem-based learning courses to online courses 
assumes that students in face-to-face and online problem-based learning courses exhibit comparable self-regulation 
of learning in the two learning environments (Savin-Baden, 2007). 

Amadieu et al. (2009) suggests that comparable self-regulation of learning among students in online 
courses is not easily achieved as students who receive instruction in online courses must overcome the cognitive 
load presented by the nonlinear nature of online courses. A similar suggestion was posited by Zumbach and Mohraz 
(2008), who indicated that students’ self-regulation of learning in online courses required students to surmount the 
cognitive load emanating from the nonlinear presentation of text. Bannert (2004) indicated that students’ self- 
regulation of learning placed a demand on students, requiring them to overcome the cognitive load generated by the 
simultaneous task of processing hypertext nodes and planning further navigation in online courses. Thus, the 
cognitive load associated with the physical structure of online courses may diminish self-regulated learning among 
students who receive problem-based learning in online courses. 

The cognitive load generated by the structure of online courses is not the only obstacle that students must 
overcome for comparable self-regulation of learning in online and face-to-face problem-based learning courses; 
students must also surmount the cognitive load inherent in problem-based learning to regulate their learning (Yuan 
et al., 2011). Ribeiro (2008) suggested that students’ self-regulation of learning requires students to overcome the 
cognitive demands of arduous workloads that are not commonly associated with learning in traditional face-to-face 
courses. International nursing students complain that these workloads are time-consuming and stressful, indicating 
that students’ self-regulation of learning is not easily achieved when problem-based learning is implemented (Yuan 
et al., 2011). Therefore, comparable self-regulation of learning among students in face-to-face and online problem- 
based learning courses may demand that students overcome the combined cognitive load associated with problem- 
based learning and the physical environment of an online course. 

Purpose of Study 

The purpose of the present study was to determine whether students’ self-regulation of learning is 
comparable in online and face-to-face problem-based learning courses. Comparable self-regulation of learning 
among students implies there should be little difference in students’ learning motivation and use of learning 
strategies. It was expected that students who received problem-based learning in an online course would differ from 
their face-to-face counterparts in learning motivation and their use of cognitive, metacognitive, and resource 
management strategies. This expected difference was based on empirical findings that suggest that the extraneous 
cognitive load associated with students’ navigation of nonlinear learning environments negatively affect students’ 
learning motivation (Christensen & Miller, 2008) and use of learning strategies (Amadieu et al., 2009). 

Furthermore, it was expected that students who received problem-based learning in an online course would differ 
from students who received problem-based learning in a face-to-face course with respect to cognitive load. This 
expectation was predicated on research that suggests that the cognitive load that students experience in problem- 
based learning is likely to be expanded by the cognitive load that students experience in online courses (Schnotz & 
Heiss, 2009). 


Research Questions 

1 . Do students who receive problem-based learning in an online course use different learning strategies (i.e., 
cognitive, metacognitive, resource management strategies) than students who receive problem-based 
learning in a face-to-face course? 

2. Does the learning motivation of students who receive problem-based learning in an online course differ 
from students who receive problem-based learning in a face-to-face course? 

3. Do students who receive problem-based learning in an online course experience different levels of 
cognitive load than students who receive problem-based learning in a face-to-face course? 

Methodology 

The present study involved a posttest-only, quasi-experimental design. All participants were selected from 
two intact nursing courses, which consisted of a face-to-face problem-based learning course and an online problem- 
based learning course. The participants in the face-to-face course served as the control group; those in the online 
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course served as the experimental group. Both groups were compared based on posttest data collected using the 
Motivated Strategies for Learning Questionnaire (Pintrich, Smith, Garcia, & McKeachie, 1991), which assessed 
participants’ self-reported motivation and use of learning strategies and the Mental Effort Rating Scale (Paas, 1992), 
which assessed participants’ cognitive load for each of four weeks following students’ completion of a weekly 
problem-solving task. Two types of statistical tests were performed on the data collection, including a multivariate 
analysis of variance and mixed analysis of variance. 

Participants 

The participants in the present study included 40 graduate nursing students at a small. Midwest Catholic 
university. Participants possessed a wide range of experience with online pedagogy, with more than half of the 
participants receiving greater than 50% of their academic program online. Although less prevalent in the 
participants’ academic program , many participants also had experience with problem-based learning, as evidenced 
by their participation in the undergraduate nursing program. All participants possessed a diverse range of technology 
skills, including competencies in the use of productivity tools, social networking sites, web-conferencing software, 
and course management systems. 

Experimental Components 

A description of the components of this quasi-experimental study is presented below. 

Treatment. The experimental group for the present study received problem-based learning in an online 
course. This course was implemented as a single module online at a distance to determine the effect of problem- 
based learning in an online course on students’ self-regulation of learning. The single module online at a distance 
approach to problem-based learning has been used extensively to assess the effectiveness of problem-based learning 
as a viable instructional framework for educating students (Lee, 2006). 

This framework was used to provide participants a weekly case study over the 4-week duration of this 
study. Participants were provided instruction on the Maastricht Seven-Step Model prior to the study to enable the 
participants, working in groups of five, to devise solutions to the issues presented in the cases. This problem-solving 
model has been used successfully to guide students’ problem-solving activities in past problem-based learning 
research (Schmidt, 1983). 

Control group. The control group consisted of the face-to-face graduate nursing course. This course was 
facilitated by the same instructor who facilitated the online version of the course and served as tutor. Participants 
received a weekly case study over the 4-week duration of the study and worked in groups of five to solve the 
problems presented in the cases. All participants received instruction on the Maastricht Seven-Step Model prior to 
initiating work on the cases. 

Problem-based learning tutor. An experienced problem-based learning instructor with prior experience 
in online instruction served as the problem-based learning tutor for both the online and face-to-face problem-based 
learning course. This individual stimulated interaction among students, provided stimuli to promote student 
elaboration, and encouraged students to integrate new knowledge with their existing knowledge. The tutor fulfilled 
these tasks by asking questions, clarifying information provided by students, and providing perspectives intended to 
enhance students’ application of knowledge. Online discussion forums and webinars supported these interactions by 
the tutor in the online course. Participants used the online discussion forums throughout the study; webinars were 
conducted on each Wednesday of the study. Students in the face-to-face course interacted with the tutor in person. 

Procedures 

Participants in the online course received a case study in an online discussion forum designated for their 
collaborative groups on Monday of each week of the study. The tutor or group members were contacted via the 
discussion forum if participants needed to clarify their understandings of the case study. Participants posted a 
definition of the problem and possible solutions to the problem in their group discussion forum prior to midnight on 
Tuesday. 

Group participants met via Adobe Connect web meeting software for approximately one hour on 
Wednesday. All groups met in virtual breakout rooms where they began one -hour meeting times in intervals (e.g.. 
Group 1 begins at 6:00 pm. Group 2 begins at 6:20 pm, and Group 3 begins at 6:40 pm) to accommodate the tutor’s 
availability. A 20-minute tutorial followed a 40-minute group meeting where an assigned scribe for each group led 
his/her group in building consensus on the nature of the problem presented, possible solutions to the problem 
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presented, priorities for solving the problem, and the knowledge base needed to solve the problem. The tutor joined 
the group after a group consensus was reached to provide coaching, clarify information, and make 
recommendations. Participants began their self-study to expand the group’s knowledge base after this meeting with 
each individual investigating a key part of the needed knowledge base. 

Participants began posting their self-study findings to the problems identified in group discussion forum on 
Thursday. All self-study postings were due at midnight on Friday. The scribe summarized the self-study postings in 
the context of the problem solution, posted the summary in the group discussion forum, and made the summary 
available to participants by Sunday at noon. Participants used the scribe’s summary to debate how the problem was 
best managed and decided whether the solution existed in the posted findings. Groups submitted a written solution 
by Monday. 

Control group. Participants met in a campus classroom on Tuesday evenings for a 3 -hour class. The tutor 
asked the students to organize themselves into groups of five and distributed a case study to all participants. All 
participants read the case study and posed questions to their group members to clarify any misunderstandings about 
the case study. The tutor was also available to clarify participant misunderstandings. Participants were prompted by 
the scribe to articulate the problem and were led by the scribe in building a consensus of the definition of the 
problem. The scribe also asked group members to propose possible solutions to the problem, establish priorities for 
solving the problem, and determine the knowledge base needed to solve the problem. All discussions were recorded 
in writing by the scribe. The tutor provided coaching, clarified information, and made recommendations. 

Self-study ocurred outside of class meetings. All participant findings were emailed to the scribe by Friday 
of each week. The scribe summarized the findings in a document and provided all group members the document of 
summarized findings by Sunday. The group debated how the findings best addressed the problem during the first 
hour of class on the following Tuesday, prior to beginning work on the next problem scenario. 

Measures 

Two sets of data were collected during this study. The first set of data consisted of repeated measures of 
students’ cognitive load over the duration of the study. This data was collected for the experimental and control 
groups using the Mental Effort Rating Scale (Paas, 1992). The second set of data consisted of posttest data collected 
using the Motivated Strategies for Learning Questionnaire (Pintrich et ah, 1991). This data was also collected for the 
experimental and control groups. 

Repeated measures. The Mental Effort Rating Scale (Paas, 1992) was used to generate weekly measures 
of participants’ cognitive load after performing a weekly problem-solving task. This instrument refers to a 9-point, 
single-item Mental Effort Scale (Paas, 1992), which was used to measure how participants rate the mental effort 
they devoted to solving the issues presented in the case study. Participants respond to the question immediately 
following the completion of the task. The scale for the Mental Effort Rating Scale (Paas, 1992) ranges from 1 to 9, 
with 1 representing very, very low effort and 9 representing very, very high effort. This subjective rating scale is used 
frequently to measure the construct cognitive load, which represents the construct mental effort (Reynolds, Woods, 
& Baker, 2007). The Mental Effort Rating Scale was administered after the completion of each weekly case study. 

Posttest. Participants in the control and experimental groups responded to the Motivated Strategies for 
Learning Questionnaire (Pintrich et ah, 1991) to determine whether students who receive problem-based learning in 
an online course differed from students who received problem-based learning in a face-to-face course with regard to 
self-regulated learning. The Motivated Strategies for Learning Questionnaire (Pintrich et ah, 1991) is an 81-item, 
self-report questionnaire that uses a 7-point Likert scale to measure college students’ motivation and use of learning 
strategies. Responses for the Likert scale ranges from 1 (Not at all true of me) to 7 (Very true of me). Two sections 
comprise the Motivated Strategies for Learning Questionnaire (Pintrich et ak, 1991): a motivation section and a 
learning strategies section. The motivation section consists of 3 1 items that consist of the expectancy, value, and 
affective components of motivation. Expectancy is divided into the learner’s control of learning beliefs and self- 
efficacy for learning and performance. The learner’s control of learning beliefs refer to the learner’s belief that 
his/her efforts to learn will result in positive outcomes. Self-efficacy for learning and performance pertains to the 
learner’s personal judgments of his/her ability to accomplish a given learning task. 

The value component of motivation is divided into intrinsic goal orientation, extrinsic goal orientation, 
and task value. Intrinsic goal orientation refers to the degree to which learners perceive themselves to be 
participating in an activity for reasons such as mastery, challenge, or curiosity. Extrinsic goal orientation pertains to 
the degree to which learners perceive themselves to be participating in an activity for competition, rewards, grades, 
performance, and evaluation by others. Task value refers to how interesting, important, and useful learners perceive 
a particular learning task to be. 
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The affective component of motivation consists of one construct: test anxiety. This construct is thought to 
be divided into two components: cognitive and emotional. The cognitive component of test anxiety refers to the 
negative thoughts that disrupt learner performance. Affective and physiological arousal aspects of anxiety comprise 
the emotional component of test anxiety. 

The learning strategies section of the Motivated Strategies for Learning Questionnaire (Pintrich et ah, 
1991) consists of 3 1 items that comprise the cognitive and metacognitive strategies rehearsal, elaboration, 
organization, critical thinking, and metacognitive self-regulation. Rehearsal relates to the recitation or naming of 
items from a list and is best used for simple tasks or the activation of information in the working memory. This 
strategy is not recommended for learners who must acquire new information. Elaboration helps learners store 
information into long-term memory by helping learners build connections to information that must be learned. 

Some elaboration strategies include paraphrasing, summarizing, generative note-taking, and the creation of 
analogies. Organization strategies help learners select appropriate information and construct connections between 
the information that must be learned. Strategies related to organization include clustering, outlining, and selecting 
the main idea from a passage. Critical thinking involves the extent to which learners report applying previously- 
learned material to a new situation to solve problems, reach decisions, or make standardized evaluations of 
excellence. 

Metacognitive self-regulation consists of three processes: planning, monitoring, and regulating. Planning 
relates to activities that activate relevant aspects of learner prior knowledge that simplify the tasks of organizing and 
comprehending the material. Such activities include the tasks of goal setting and task analysis. Monitoring 
activities involve learner tracking of attention, self-testing, and questioning. These activities help the learner 
understand new material and integrate the new material with prior knowledge. Regulating activities supports the 
fine-tuning and continuous adjustment of one’s cognitive activities. Such activities are intended to improve 
performance by helping learners to check and correct their behavior as the learners work toward a given learning 
goal. 

The learning strategies section of the Motivated Strategies for Learning Questionnaire (Pintrich et al., 

1991) further includes 19 items that involve resource management strategies. Resource management strategies 
include such strategies as time and study environment management, effort regulation, peer learning, and help- 
seeking. Time and study environment strategies pertain to the learner’s capacity to manage study time and the 
learning environment. These strategies require learners to manage their study time by scheduling and planning 
activities. Learners must also regulate the study environment by ensuring that the study environment is quiet, 
organized, and free of visual and auditory distractions. Effort regulation strategies refer to the learner’s capacity to 
control his/her effort and attention when presented with distractions or uninteresting material. These strategies are 
not only an important aspect of goal commitment but also foster the continued use of self-regulated learning 
strategies. Peer learning involves learner collaboration with peers. Such collaboration has been shown to have 
positive effects on achievement, helping learners to clarify course material and attain insights not easily achieved in 
isolation. Help-seeking refers to strategies that learners use to seek academic help. Such strategies require the 
learner to be able to determine that help is needed and seek help from the instructor or peers. 

Results 

The data collected for the present study allowed the researcher to address the three research questions. 
Results for Research Question 1 are described in the first section to present statistical analyses of data pertaining to 
student use of self-regulated learning strategies in online and face-to-face courses where problem-based learning is 
the exclusive method of instruction. This section is followed by a description of the data analyses for Research 
Question 2, which describes students’ learning motivation in online and face-to-face courses where problem-based 
learning is the exclusive method of instruction. The third section describes the data analyses for Research Question 
3. Data analyses for this section present a comparison of cognitive load data for students in online and face-to-face 
problem-based learning courses. 

Do students who receive problem-based learning in an online course use different learning strategies than 
students wbo receive problem-based learning in a face-to-face course? 

A multivariate analysis of variance was performed to test the hypothesis that students who received 
problem-based learning in an online course used different self-regulated learning strategies than students who 
received problem-based learning in a face-to-face classroom. Student use of self-regulated learning strategies was 
assessed by nine measures: rehearsal, elaboration, organization, critical thinking, help seeking, peer learning, time 
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and study environment, effort regulation, and metaeognitive self-regulation. These measures were assessed for 
students at both levels of the independent variable, as shown in Table 1 . 


Table 1 

Mean Scores and Standard Deviations for Measures of Student Use of Learning 


Strategies 


Strategy 

M 

F2F 

SD 

- 

Group 

Online 
M SD 

- 

REH 


4.75 

.444 


4.61 

.349 

ELA 


2.18 

.099 


2.16 

.072 

ORG 


4.65 

.454 


4.69 

.302 

CTK 


4.85 

.440 


4.62 

.289 

MSR 


5.12 

.424 


4.68 

.364 

TSE 


4.83 

.340 


4.81 

.345 

ERE 


4.92 

.354 


4.79 

.374 

PEN 


4.98 

.439 


4.97 

.458 

HSK 


4.95 

.350 


4.75 

.303 


Note. REH = rehearsal; ELA = elaboration; ORG = organization; CTK = eritieal thinking; MSR = metaeognitive 
self-regulation; TSE = time and study environment; ERE = effort regulation; PEN = peer learning; HSK = help 
seeking. 


A statistieally signifieant multvariate effeet was obtained, F (9, 30) = 11 .243, < .0005; Wilks' A = .229; 
partial = .771. The multivariate effeet size indieated that student use of learning strategies aeeounted for 77.1% 
of the varianee in the eanonieally derived dependent variable. 

Follow-up univariate analysis of varianee (ANOVA) tests were eondueted to test the assumption of 
homogeneity of varianee for eaeh of the nine learning strategies’ subseales. A series of Levene’s F tests indieated 
that the homogeneity of varianee assumption was satisfied (p > .05). A series of post-hoe analyses (Tukey) were 
performed to examine individual mean differenee eomparisons aeross both levels of problem-based learning 
environment and all nine learning strategies’ subseales. The results revealed that the post-hoe mean eomparison for 
the learning strategy metaeognitive self-regulation was statistieally signifieant (p < .05), suggesting that students 
who reeeived problem-based learning in a faee-to-faee eourse had a statistieally signifieantly higher mean seore on 
the dependent variable metaeognitive self-regulation (7^(1, 38) = 12.004,/? < .005; partial tf = .240) than students 
who reeeived problem-based learning in an online eourse. The effeet size of .240 indieated that metaeognitive self- 
regulation aeeounted for 24% of the varianee. Therefore, the hypothesis for Research Question 1 was eonfirmed. 

Does the learning motivation of stndents who receive problem-based learning in an online conrse differ from 
stndents wbo receive problem-based learning in a face-to-face conrse? 

A multivariate analysis of varianee was performed to test the hypothesis that the learning motivation of 
students who reeeived problem-based learning in an online eourse differed from students who reeeived problem- 
based learning in a faee-to-faee eourse. Students’ learning motivation was assessed by six measures: intrinsie goal 
orientation, extrinsie goal orientation, eontrol of learning beliefs, task value, test anxiety, and self-effieaey for 
learning and performanee. These measures were assessed for students at both levels of the independent variable, as 
shown in Table 2. 
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Table 2 


Mean Scores and Standard Deviations for Measures of Learning Motivation 


Strategy 

M 

F2F 

SD 


Group 

Online 
M SD 


IGO 


4.69 

.420 


4.62 

.349 

EGO 


4.60 

.462 


4.52 

.371 

COL 


4.80 

.402 


4.74 

.401 

SEF 


4.82 

.359 


4.70 

.307 

TVA 


4.86 

.486 


4.62 

.349 

TAX 


4.72 

.412 


4.67 

.339 


Note. IGO = intrinsic goal orientation; EGO = extrinsic goal orientation; COL = control of learning; SEE = self- 
efficacy for learning and performance; TVA = task value; TAX = test anxiety. 


No statistically significant multivariate effect was obtained, F {( 3 , 33) = 2.\22,p = .07; Wilks' /I = .722; 
partial rj2 = .278. This finding suggests that students who received problem-based learning in an online course did 
not differ from students who received problem-based learning in a face-to-face course with respect to learning 
motivation. Thus, the hypothesis for Research Question 2 was not confirmed. 

Do students who receive problem-based learning in an online course experience different levels of cognitive 
load than students who receive problem-based learning in a face-to-face course? 

A mixed-model analysis of variance test was performed to determine whether the self-reported cognitive 
load of students who received problem-based learning in an online course differed from students who received 
problem-based learning in a face-to-face. Cognitive load scores for weeks 1 through 4 were the within-subjects 
variables with time as the within-subjects factor name. The between-subjects factor was problem-based learning 
environment. A 4 (Time) x 2 (Problem-based learning environment) mixed-model ANOVA with overall scores 
revealed no statistically significant interaction between the intervention and time on cognitive load, F (3,1 14) = 
.063,/? = .979, partial if = .002. The main effect of time showed no statistically significant difference in cognitive 
load, F (3,114) = .063, = .979, partial i]^ = .002. This finding indicated that there was no difference in cognitive 
load regardless of the intervention group. The main effect of group showed no statistically significant difference in 
cognitive load between intervention groups, F (1, 38) = .073,/? = .789, partial if = .002. This finding indicated that 
there was no statistically significant difference in cognitive load between the different intervention groups. 
Therefore, the hypothesis for Research Question 3 was not confirmed. The findings on the dependent variable are 
consistent with a significant disordinal interaction, as shown in Figure 1 . 



Figure 1. Level of cognitive load for online and face-to-face groups after each of four weeks. The figure shows a 
significant disordinal interaction. 
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Group means and standard deviations for four measures of eognitive load are shown in Table 3. 


Table 3 


Group Mean Scores and Standard Deviations for Measures of Cognitive Load 


Week 

F2F 

M 

SD 

Group 

Online 

M 

SD 

1 

6.75 

1.21 

6.80 

1.15 

2 

6.65 

1.14 

6.80 

1.24 

3 

6.85 

1.14 

6.80 

1.20 

4 

6.75 

1.25 

6.85 

1.14 


Discussion 

The environment in whieh students reeeive problem-based learning appears to impaet their learning 
proeess. An examination of Research Question 1 revealed that students who reeeived problem-based learning in an 
online eourse did not use self-regulated learning strategies in a manner eomparable to students who reeeived 
problem-based learning in a faee-to-faee eourse. The primary eonhibutor to this differenee was the variable 
metaeognitive strategy use. This finding seems reasonable in light of findings in eomputer-supported eollaborative 
learning literature, whieh indieates a laek of student partieipation in the eo-regulation of learning (Oehoa & 
Robinson, 2005; Rose, 2004). One implieation of this literature is that students who reeeive problem-based learning 
in an online eourse would not be expeeted to use metaeognitive strategies as effeetively as their faee-to-faee 
eounterparts as students who reeeive problem-based learning in an online eourse are typieally inelined to eoneede to 
minority opinions with little eognitive dissonanee or eontribution (Oehoa & Robinson, 2005). Hung and Crooks 
(2009) offered an explanation for this behavior, reporting that eollege students regarded online eonneetions to 
eollaborative group members as superfluous, ineonvenient, and unsupportive of their learning proeess. This attitude 
would be expeeted to impaet students’ motivation to eo-regulate learning. 

Furthermore, students who reeeived problem-based learning in an online eourse did not exhibit different 
learning motivation than students who reeeived problem-based learning in a faee-to-faee course. Both groups of 
graduate students exhibited high learning motivation, as determined by the Motivated Strategies for Learning 
Questionnaire (Pintrich et ah, 1991). This finding is consistent with Hung et al. (2010), who reported students’ level 
of education positively influenced students’ learning motivation. On the other hand, the finding that the learning 
environment had no differential effect on students’ learning motivation departs from existing literature that suggests 
that students’ learning motivation is influenced by the learning environment (Bolliger, Supanakom, & Boggs, 2010). 

Moreover, students who received problem-based learning in an online course not only exhibited learning 
motivation comparable to their face-to-face counterparts but also experienced a comparable level of cognitive load 
in comparison to their face-to-face counterparts. Students in both groups experienced an excessive level of cognitive 
load. This observation is consistent with past research, which has shown that cognitive load is typically high for 
students who receive any form of problem-based learning (Ruiz-Gallardo et ah, 2011; Yuan et al., 2011). 
Conversely, the observation of comparable cognitive load among these students is not consistent with past research 
that suggests that students who receive instruction in online courses experience a different level of cognitive load 
than students who receive traditional classroom instruction (Amadieu et al., 2009; Banner!, 2004; Schnotz & Heiss, 
2009). In conclusion, it cannot be determined from the present study whether excessive cognitive load is the result 
of instruction or the learning environment but high cognitive load among students who receive problem-based 
learning is in line with the established literature. 

Implications for Practice 

Results of the present study indicated that there is a disparity in metaeognitive sfrategy use between 
students who receive problem-based learning in online and face-to-face courses. Students who received problem- 
based learning in an online course were less likely to use metaeognitive sfrategies to regulate learning than their 
face-to-face counterparts. This finding seems to indicate that there was a lack of support for metaeognitive strategy 
use in the online course. Past research has shown that students’ metaeognitive strategy use is improved by the tools 
(discussion forums and email) used to support learning in an online course (Dabbagh & Kitsantas, 2005). This 
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finding is consistent with Kramarski and Mizrachi (2006), who reported that discussion forums with metacognitive 
guidance helped students in online courses surmount difficulties associated with the metacognitive tasks of goal 
setting, planning, self-monitoring, and evaluation. Thus, students need support for metacognitive self-regulation 
when problem-based learning is implemented in an online course. 

A second implication of the results of the present study is that efforts should be made to reduce cognitive 
load when problem-based learning is implemented in online and face-to-face courses. While the influence of 
cognitive load on students’ self-regulation of learning was not assessed in the present study, the level of cognitive 
load was determined to be excessive, as defined by Paas (1992). Morozov (2007) showed that the high cognitive 
load that students experienced could be reduced through the use of a comprehensive hierarchical map to help 
students navigate the learning material. Research by Schnotz and Heiss (2009) suggested that cognitive load could 
be reduced through the use of semantic scaffolds. A study by Cheon and Grant (2012) demonstrated that cognitive 
load could be reduced through the use of a metaphorical interface, which contributes to schema automation and 
construction. Therefore, the high level of cognitive load experienced by students in the present study should be 
addressed through instructional design of the learning environment. 

Future Research 

The present study showed that students who received problem-based learning in an online course exhibited 
statistically significant differences from students who received problem-based learning in a face-to-face course with 
respect to student use of self-regulated learning strategies. It was found that group differences in metacognitive self- 
regulation contributed to a multivariate main effect. The limited use of metacognitive strategies by students who 
received problem-based learning in an online course indicated that students who received problem-based learning in 
an online course needed support for metacognitive strategy use. 

A study by Azevedo et al. (2005) revealed that adolescents who received adaptive human scaffolding used 
more metacognitive strategies in the regulation of their learning process. This form of scaffolding has been 
consistently shown to support metacognitive monitoring through the capacity of human tutors to continuously 
monitor learning progress, diagnose students’ emerging understanding, and provide students’ timely scaffolding 
(Aleven & Koedinger, 2002). Thus, research on human adaptive scaffolding in online and face-to-face problem- 
based learning courses might determine whether human adaptive scaffolding eliminates the disparity in 
metacognitive strategy use between students who receive problem-based learning in an online course and students 
who receive problem-based learning in a face-to-face course. 

Other forms of scaffolding might also eliminate the disparity in metacognitive strategy use between 
students who receive problem-based learning in an online course and their face-to-face counterparts. Van den Boom, 
Paas, and Merrienboer (2004) reported that students who received reflection prompts in tandem with tutor feedback 
demonstrated improved metacognitive strategy use. Similar improvements in students’ metacognitive strategy use 
were reported by Moos and Azevedo (2008) when students were afforded conceptual scaffolding. Therefore, 
research on different types of scaffolding is needed to determine how students who receive problem-based learning 
in an online course are best supported in the regulation of learning. 

In addition, research on the use of learning diaries is recommended for future studies that compare 
students’ self-regulation of learning in online and face-to-face problem-based learning courses. Kuusela and Paul 
(2000) indicated that diaries enable an examination of the nature of students’ thinking and reasoning through 
retrospective verbal protocol analysis. Such analysis of diaries allows researchers to capture students’ self-regulation 
of learning across a given learning task through students’ generation of narrative content (Ericsson & Simon, 1993). 
Capturing students’ self-regulation of learning across a particular learning task through the use of learning diaries 
would likely detect changes in self-regulated learning not apparent when self-regulated learning is reported via self- 
report questionnaire. 

Moreover, conducting the present study using learning diaries would capture students’ self-regulation of 
learning and cognitive load around the different phases of problem-based learning as opposed to the reporting phase 
of problem-based learning. Schmitz and Wiese (2006) indicated that diaries structured using a series of event 
questions captured students’ self-regulation of learning before and after events. Such event questions would likely 
make it possible to assess cognitive load and self-regulated learning at the different stages of problem-based 
learning. The present study did not examine self-regulated learning and cognitive load with respect to the events of 
problem-based learning. As such, the present study may not have fully accomplished its intent. Thus, subsequent 
studies of this nature may be best conducted with a focus on the events of problem-based learning. 
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Limitations of Study 

The quasi-experimental design of the present study limits its generalizability. One issue with the design of 
this study was the lack of participant randomization, which introduced several confounding variables, including 
participants’ personal characteristics, technology efficacy, and technology attitudes. Such confounding variables 
pose a serious threat to the external validity of this study, limiting the researcher’s ability to make sound inferences 
from the data. 

A second limitation of the study involved the population from which the participants were selected. All 
participants were graduate nursing students with one or more years of clinical experience. Results for this group may 
not be easily generalized to undergraduate nursing students and students in academic disciplines where less 
academic rigor is evident. 

A lack of standardization in course design represented a third limitation of this study. The designer of the 
learning environments used in the present study was accustomed to making arbitrary course-design decisions. Most 
of the course designer’s decisions reflect personal preferences as opposed to sound instructional design practices. 
Therefore, the results of this study cannot be generalized to situations where best instructional design practices have 
been applied. 

The use of the Mental Effort Rating Scale presented a fourth limitation of this study. This instrument only 
allowed the researcher to determine whether students experienced high cognitive load after performing a learning 
task. It is impossible to know whether cognitive load is a result of the learning environment or the instruction. 
Therefore, the results of this study are not easily generalized to other situations. 
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Academic Technology (AT) is a fast growing field that deserves attention given its dynamic nature and 
impact on educational practices. The field has evolved from information technology to concentrate on advancing 
technology to enhance teaching and learning. Yet, the field appears to be insufficiently mature or defined making it 
difficult for AT professionals to be categorized and characterized or to fully understand their changing roles. There 
is uncertainty as to the roles, responsibilities and positions of AT professionals both within and outside of the field. 

An online survey was conducted of 8 1 AT professionals to better understand the responsibilities and 
perceptions of their roles, concerns and influence. The study findings concludes that the majority of AT 
professionals have influence on the AT decision making process at their institution of higher learning, AT decisions 
appear to be made based on technological rather than pedagogical considerations and AT professionals have an 
expanded responsibilities and obligations at their institution of higher learning. 

Introduction 

For more than 200 years, colleges and universities have embraced digital technology and employed it in 
support of virtually all aspects of academic and administrative college life (Oblinger, 2010). Nevertheless, Susan 
Metros, Associate Vice Provost and Deputy Chief Information Officer (CIO) at the University of Southern 
California, describes a troublesome situation that plays out on college campuses all over the world: 

IT [Information Technology] departments are often guilty of offering services that are technically complex, 
user unfriendly, poorly communicated, and perceived as changing too rapidly. Many IT systems are based 
on technical requirements decided by the IT group alone and not on what is best suited for the faculty 
member undertaking research, advancing scholarship, teaching classes, and/or serving the community. 
(Metros, 2010, p. 54) 

Typically, colleges have one IT department to service a wide range of goals and needs, even though there 
can be no single technological approach that adequately addresses all of them. If the goal is to support teaching and 
learning at the college, the focal issue of this paper, IT personnel who understand technological applications to 
teaching should make a dedicated effort to communicate and work with faculty and students to achieve the college’s 
mission (Nworie & Albright, 2008). This bridge between the IT and teaching worlds is Academic Technology (AT), 
usually led by the Senior AT Officer (Nworie & Albright, 2008). AT professionals are found on campus assisting IT 
departments and supporting faculty in integrating technology into their teaching repertoire in a pedagogically sound 
way, yet they are seldom found in influential positions on campus (Kowch, 2005). 

Paul Michael Privateer, Professor of Humanities at Arizona State University, states that if colleges are 
going to be contemporary and effective they must have a strategic AT agenda that focuses on changing the model of 
traditional higher education, where the emphasis has been on storage and recall of infomiation rather than producing 
intelligence, enabling students to synthesize information, and linking it to real world situations (Privateer, 1999). He 
further states that higher education needs to develop a strategically guided approach to technology-mediated 
instruction. In order for this to happen, learning outcomes need to be consistent and integrated throughout the 
curriculum, and there should be dedicated resources on campus that are managed by an AT officer (Privateer, 1999). 

Common examples of AT are Learning Management Systems (LMS), used for hybrid and distance learning 
courses; Student Response Systems (Clickers), used for in-class assessment; and video podcasting, commonly used 
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in flipped classroom scenarios. These technologies are just a few examples of those used in teaching and learning in 
higher education. The list of technologies that are currently used and will be used in the future is constantly 
changing and growing. 

In different forms, technology has been a part of education for a long time. Michael J. Albright (1989) 
wrote “Instructional Technology has never had a brighter future than in 1989,” yet Susan Metros’ comment reflects 
the ambiguousness of AT in 2010. In the 1 1 years between Albright and Metros’ comments, individuals filled AT 
roles on campus, but many were without clear charters and official, defined positions within their organizations 
(Nworie, 2005). 

Ana Donaldson, past president of the Association for Educational Communications and Technology, wrote 
that describing the profession of AT to others can be a daunting task. Part of the challenge that AT professionals 
face is even what to call themselves, and Donaldson suggests that if position titles were more descriptive and 
uniform it would be easier to clarify what the professional does (Donaldson, 2012). John Nworie (2005) also states 
that as the roles and responsibilities of the AT professional changes, so does the job title. 

The terminology used to describe the field needs to be clear. Johnson et.al. (2003) broadly define AT 
professionals as those who encompass the broad functions of creating collections of digital resources for faculty and 
student use; designing and supporting classroom technologies; and assisting faculty to integrate technology into their 
teaching. The lack of a clear understanding of the AT field leads to much confusion when discussing the roles and 
responsibilities of the professionals who identify with the community. According to Bates and Sangra (2011), 
academic technologists should be responsible for helping the college support innovation in teaching and establish 
goals and strategies for learning technologies. These AT professionals should facilitate a collective approach to 
setting and implementing learning technology goals and be champions for change in the way that instructors teach 
(Bates & Sangra, 2011). 

Information Technology (IT) focusses on technology implementation and support across campus, while AT 
concentrates on those technologies that support teaching and learning and the pedagogy of their implementation into 
the curriculum. As Table 1 illustrates, IT professionals tend to focus on technology as a tool, ensure its delivery, and 
maintain the infrastructure. They work with an end user to ensure that person can make the technology tool function. 
The AT professional emphasizes technology as an application to achieve pedagogic goals and objectives, focuses 
on integration of technology and instructional design and the creation of content and methods that are appropriate 
for technology use in education. 

Table 1 


A comparison of Information Technology and Academic Technology 


Information Technology 

Academic Technology 

Technology as a Tool 

Technology as an Application 

Focus on Delivery 

Focus on Integration 

Technology Infrastructure 

Instructional Design (Content Production) 

End User Support 

Faculty Development 

Systems and Network Administration 

Technical and Pedagogic Training 

Accessibility 

Skills Development 

Stability 

Quality Assurance and Assessment 


The 2011 CDW-G Report (Caraher & Braselman, 2011) states that there is a disconnect between the 
priorities of college administrators and the way in which technology is leveraged in teaching and learning at their 
institutions. In theory, a college’s different operating divisions are united in fulfilling the mission to educate 
students, yet the technology employed in support of one unit is often at odds with that used in another. Leadership is 
key to the success of a learning community (Wenger, White, & Smith, 2009), and the communal aspects of 
technology imply that it will help find learning partners and engage them meaningfully (Wenger et al., 2009). 
Wenger et al. (2009), state that while integration of technology into education is an important technical goal, there 
are limits to what we can expect directly from technology. Ideally, technology must be applied in a meaningtul way 
to create a relationship between the tool and the educational goal (Wenger et al., 2009). AT professionals on campus 
are in a position to achieve this relationship, but they are rarely in leadership or influential positions, so despite the 
importance of well-adapted technology, a comprehensive, integrated view of AT’s role in teaching and learning is 
often missing. 
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study Overview 


The value of technology rests in the quality and effectiveness of the activities that it supports. Technology 
in education is not a new phenomenon, but the speed at which technology changes and different technologies are 
introduced into education is constantly increasing. The AT field directly supports technologies that are used 
purposefully to enhance teaching and learning in higher education. Colleges and universities are making substantial 
investments in technology platforms to achieve their mission of educating students to have meaningful and 
productive lives in the workforce and in the community at large (Arroway & Sharma, 2009; Green, 2009). 

IT departments, under the leadership of the CIO, often make decisions regarding technology investments 
and usage based on non-academic criteria that are not always attuned to the needs of faculty and students. AT 
professionals are the bridges between the administrative departments (including IT) and the academic realm on 
campus. These individuals are versed in both technology and academic fields and are able to move comfortably in 
both sectors. However, many higher education institutions do not have an organized AT strategy that extends across 
campus, nor do they have AT professionals with sufficient influence to implement campus-wide improvements. 
Contributing to the problem are the dynamic nature of technology use in teaching and learning and a broadly defined 
AT field, making it difficult for some AT professionals to easily explain what they do (Donaldson, 2012). 

This study sought data to clarify the position of AT professionals, their roles, and their ability to establish 
priorities and policies for integrating technology into teaching and learning. Specifically of interest was whether or 
not AT professionals can influence decisions that set strategic goals and strategies for learning technologies, allocate 
resources, approve projects and evaluate the effectiveness of technology strategies at their institutions (Bates & 
Sangra, 2011). 

This study ascertained the responsibilities of AT professionals based on what they self-report their duties 
are and what they believe their responsibilities should be to identify perceived gaps in responsibilities and how they 
affect the influence that AT professionals currently have on college campuses. 

This research is a descriptive study based on factor analysis methodology. Descriptive research questions, 
such as those in this study, are asked in order to describe a situation at a specific point in time. This descriptive study 
summarizes the current status of AT professionals in higher education, a valuable and needed first step to investigate 
this area (Mcmillan & Schumacher, 2010). 

The goals of this research were twofold: 

1. To create a snapshot of AT professionals’ current roles and responsibilities in higher education and the 
perceived influence that they have at their institution. 

2. To gain an understanding of what AT professionals believe their roles, responsibilities, and influence 
should be at their institution. 

The goals of the study were to create a snapshot of the current roles and responsibilities of AT 
professionals in higher education and to understand what they believe their responsibilities and influence should be. 

To achieve these goals the following six research questions were asked: 

1 . What are the major areas of responsibilities of AT professionals? 

2. To what extent do AT professionals believe they fulfill their perceived level of obligations across the areas 
of responsibilities determined in Research Question 1? 

3. Is there a gap in performance across the areas of responsibilities? 

4. Is there a relationship between the magnitude of the gap in performance in obligations and the professional 
background and characteristics of the AT professionals? 

5. To what extent do AT professionals believe that they participate in institutional AT decision making? 

6. Is there a relationship between the magnitude of the gap in performance and the degree of perceived 
participation in decision making? 

In addition to other anticipated outcomes of this study, the conclusions may assist in addressing ambiguities 
in the field as it is currently constituted, namely, the lack of precision and clarity in delineating the roles and 
responsibilities of AT professionals. The results of this study can also help clarify roles and expectations and thereby 
improve the field. Further, this study will assist those who want to work, and institutions that want to hire people to 
work, in the field of AT. 
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Subjects and Sample 

The subjects of this study are the perceptions of current AT professionals in higher education whose 
primary function is to serve their academic community — faculty and students — ^by creating technology-enhanced 
pedagogical experiences. These individuals hold various titles that include, but are not limited to, AT directors, 
instructional designers, and directors of eLearning. They are found in technology centers on campus and within 
various departments, including IT and the provost’s office. Regardless of the differences in titles and departmental 
affiliation, since these individuals identify themselves as belonging to the AT profession, we were interested in 
gathering information about how they perceive their own roles and responsibilities. 

Individuals were contacted through organizations that cater to the AT profession: Educause Learning 
Technology Leadership Alumni distribution list (LTLAlum); and the Association for Educational Communications 
and Technology (AECT). LTLAlum has 375 people on their email distribution list and AECT has 2,200 members 
in good standing. Both organizations publish journals, provide professional development, and hold conferences. 

The principle researcher used email to contact the manager of the LTLAlum distribution list and the person 
responsible for AECT research initiatives. Both responded by email agreeing to disseminate the survey. 

Instrumentation and Data Collection 

Descriptive research design uses a data-gathering instrument to obtain numerical indices that correspond to 
characteristics of the subjects, a process that provides objectivity in measuring and describing phenomenon by use of 
numbers and statistics to explain, predict, or describe a situation (Johnson & Christensen, 2008; Mcmillan & 
Schumacher, 2010). These numerical values are then summarized and reported as results. 

For this study, data were collected with a survey of the type used frequently in education because it 
obtained accurate information for large numbers of people with a small sample. The instrument included the same 
questions for all participants from the sample target population of AT professionals selected by the investigator 
(Mcmillan & Schumacher, 2010). 

Validity 

Validity is the extent to which inferences made on the basis of numerical scores are appropriate, meaningful, and 
useful (Mcmillan & Schumacher, 2010). Content validity was established using a panel of three experts. Each of the 
experts holds an Ed.D. and is knowledgeable about a different area of the AT field. A modified copy of the survey 
was created on the web-based survey tool “Novi Survey.” The panel members were asked to evaluate whether the 
survey questions adequately address the research questions. Majority rule methodology was applied to the 
recommendations of the panel. 

Reliability 

Instrument reliability refers to the consistency of measurement (Mcmillan & Schumacher, 2010). The Cronbachs 
Alpha test was used to establish reliability and internal consistency of the survey questions. The test was mn on the 
does perform and should perform selections of questions 8 through 33. Cronbachs Alpha established that 93% of 
the variance in the composite score in internally consistent and reliable (Table 2). 

Table 2 


Cronhach's Alpha Reliability Statistics 


Cronbach's Alpha 

Cronbach's Alpha 
Based on 
Standardized Items 

N of 
Items 

.932 

.936 

60 


Data Collection 

After the survey was vetted by the experts, the questions were entered into the web-based survey tool 
“Novi Survey.” According to Mcmillan & Schumacher (2010), online surveys offer advantages, compared to other 
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techniques, such as quick response, easy follow-up, and the ability to survey a large population. The survey was 
designed to be as short as possible to achieve the maximum response rate from the subjects. 

The survey was available online for 2 weeks. The survey instrument had seven multiple-choice and short- 
answer questions to ascertain current employment and background characteristics. These were followed by 30 
statements describing different professional responsibilities. Participants were asked to respond to these questions in 
two different ways. The first response reflected whether or not they do perform the task. The second response 
reported the respondent’s opinion whether or not they should perform the task. Both sets of responses were based on 
a Likert scale ranging from 1 to 5, with 1 representing “never” and 5 representing “always.” 

The last four statements of the survey were designed to ascertain the level of influence the respondents do 
have and believe they should have at their institutions. The recipients of the instrument were asked to respond to 
these questions in two different ways. The first response reflected whether or not they agree that they have 
influence. The second response is the respondent’s opinion as to whether or not they should have influence. Both 
sets of answers were again based on a Likert scale ranging from 1 to 5, with 1 representing “agree” and 5 
representing “disagree.” Table 3 shows examples of the survey questions. 

Table 3 


Sample Survey Questions 



Please rate these questions to the following scale: 

1 = Strongly disagree, 2 = Disagree, 3 = Neither agree nor disagree, 4 = Agree, 5 = Strongly agree 
Please circle your choice in both columns for each item. 

DOES PERFORM SHOULD PERFORM 

agree disagree agree disagree 

(34) 

1 

2 

3 

4 

5 

I have the opportunity to express my perspective 
or make recommendations for a particular cause 
of action. 

1 

2 

3 

4 

5 

(64) 

(35) 

1 

2 

3 

4 

5 

I believe my recommendations effect the decision 
making process at my institution. 

1 

2 

3 

4 

5 

(65) 


Results 

A factor analysis study was conducted of individuals who describe themselves as AT professionals. 
Seventy-one AT professionals completed the online survey. The majority of the respondents work at a 4-year 
baccalaureate college and hold a master’s degree. About % of the respondents have an employment background in 
education, about half in teaching. Many of the professionals do not have a position title that fit within the survey 
choices. Approximately half of the respondents are considered staff connected to the IT unit of their institution. 
Less than a quarter of these individuals are affiliated with the academic executive branch or Provost Office. 

Summary of RQ 1 : Major responsibilities of AT professionals 

The first research question of this study: “What are the major areas of responsibilities of AT 
professionals?” was asked in order to categorize the roles, responsibilities and obligations of AT professionals. A 
factor analysis of the data, accounting for 97% of the variation in data, revealed that AT professionals primarily 
have responsibilities and obligations in the following five categories: 

1 . Strategic Planning 

2. Instructional Design 

3. Ongoing Personal Professional Development 

4. Academic Technology Management 

5. Research and Assessment 

Summary of RQ 2: Fulfillment of responsibilities 

The second research question: “To what extent do AT professionals believe they fulfill their 
perceived level of obligations across the areas of responsibilities determined in Research Question 1?” sought to 
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examine the extent that the respondent AT professional fulfills the responsibilities and obligations defined in 
researeh question one. The survey asked the respondents to indieate the level to whieh they “do” perform a speeifie 
responsibility and the level whieh they feel they “should” perform the responsibility. The average seore of the “do” 
and “should” responses were ealeulated. Where the mean seore of the “should” response was greater than the “do” 
response, the respondent believed that they responsibility should be fulfilled to a greater extent than it is eurrently. 
This was the ease with all of the responsibilities, with the greatest differenees in Ongoing Personal Professional 
Development and Academic Technology Management. In other words, though AT professionals believe they should 
fulfill all of the responsibilities more than they do, this is espeeially so in the areas of Ongoing Personal 
Professional Development and the Academic Technology Management. 

Summary of RQ 3: Gap in Performanee of Responsibilities 

As researeh question 2 showed, there is a differenee in fulfillment of responsibilities by AT professionals. 
Researeh question 3 : “Is fhere a gap in performanee aeross the areas of responsibilities?” sought to identify 
pereeived gaps in those responsibilities and obligations. A statistieal paired t-test was eondueted to determine if 
there were any signifieant gaps between the levels that responsibilities were performed. It was determined that there 
was a signifieant gap in performanee aeross all the stated responsibilities and obligations that AT professionals 
perform. 

Summary of RQ 4: Relationship of performanee gap and respondent eharaeteristies 

After it was determined that there were signifieant gaps in the fulfillment and performanee of AT 
responsibilities, researeh question 4: “Is there a relationship between the magnitude of the gap in performanee in 
obligations and the professional baekground and eharaeteristies of the AT professionals?” identified the magnitude 
of the gaps and eompared it to the eharaeteristies of the respondent. It was diseovered that only two respondent 
eharaeteristies, institutional unit connected to and official title, influeneed the performanee gap of responsibilities. 
The performanee of “Strategie Planning” responsibilities was influeneed by the intuitional unit connected to with 
ATP’s eonneeted to the Media Center reporting the greatest underperformanee and those eonneeted to IT the least 
underperformanee. The official title of an AT professional effeeted the performanee level of “Instraetional Design,” 
“Ongoing Personal Professional Development,” “Aeademie Teehnology Management” and “Researeh and 
Assessment” responsibilities. 

AT professional’s with the offieial title oi Academic Technology Director and eLearning Director report 
that they underperform in the areas of “Instruetional Design” and “Researeh and Assessmenf’ and over partieipate in 
“Ongoing Personal Professional Development.” Those with the title of Instructional Designer report that they over 
perform in the areas of “Instruetional Design” and “Researeh and Assessmenf’ and over partieipate in “Ongoing 
Personal Professional Development.” 

Summary of RQ 5: Extent of partieipation in AT deeision making 

Researeh question 5: “To what extent do AT professionals believe that they partieipate in institutional AT 
deeision making?” It was found that the majority of respondents agree that they have an opportunity to express their 
reeommendation for a partieular eourse of aetion and that these reeommendations effeet the deeision making proeess 
at their institution. 

Summary of RQ 6: Relationship between gap in performanee and AT deeision making 

Finally the study sought to answer researeh question 6: “Is there a relationship between the magnitude of 
the gap in performanee and the degree of pereeived partieipation in deeision making.” It was found that the stronger 
pereeption the respondents had that they have the opportunity to express their perspeetive and make a 
reeommendation for AT deeision making, the less likely there were to over perform in the areas of “Strategie 
Planning” and “Instruetional Design” and they are moderately likely to over partieipate in “Ongoing Personal 
Professional Development” and over perform “Researeh and Assessmenf’ responsibilities. 
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Conclusions 


Relationships and trends were identified in the resulting data, and three eonelusions beeame evident: a) AT 
professionals underperform Aeademie Teehnology Responsibilities b) AT deeisions appear to be made based on 
teehnologieal rather than pedagogieal eonsiderations; and e) AT Professionals have the opportunity to express their 
opinions and influenee AT deeision making at their institution. These eonelusions have an effeet on the field of AT 
as a whole, the individuals who are working in the field, and those who are affeeted by the AT deeisions that are 
made. 

AT Professionals Underperform Aeademie Teehnology Roles 

In this study the researeh results show that AT professionals (ATP’s) perform their responsibilities and 
obligations less than they believe that they should be performing them. Researeh questions 2 and 3 eoneentrated on 
the extent to whieh AT professionals fulfdl their obligations and responsibilities and whether or not there is a gap in 
their performanee and if the gap in performanee is relational to the professional baekground eharaeteristies of the 
ATP. The average seore of the responses to the “do” and “should” statements were ealeulated and the mean, mode, 
median and standard deviation was determined. The results show that aeross all of the responsibilities and 
obligations of ATP’s there is a feeling that they should be performing them more than they eurrently are performing. 

AT is an integral and essential eomponent of almost all eore higher edueation aetivities and needs to be 
managed as sueh (Bates & Sangra, 2011). ATP’s tend to be eonsidered supervisors in both the administrative and 
the aeademie domains of a eollege or university. On one hand they support institutional effieieney and eontinually 
monitor the eost-effeetiveness of the applied teehnology, while on the other, they maintain the quality of the 
aeademie program and the proper applieation and implementation of teehnology in the eurrieulum and teaehing on 
eampus (Sellers, 2005). 

As AT has expanded in higher edueation, so too has the responsibilities of the ATP. Those responsibilities 
now inelude Strategic Planning, Instructional Design, Ongoing Personal Professional Development, Academic 
Technology Management and Research and Assessment. ATP’s believe that they underperforming their 
professional responsibilities. It seems that the expeetations of the institution have expanded the seope of the ATP 
and there is not enough time and resourees to perform the funetions that the ATP feels is neeessary. 

AT Deeisions Appear To Be Made Based On Teehnologieal Rather Than Pedagogieal Considerations 

In this study the researeh data show that the Aeademie Teehnology organization stmeture favors the 
teehnologieal side of the institution rather than the pedagogie side. ATP’s are attaehed to the Information 
Teehnology departments, under the leadership of the Chief Information Offieer (or Chief Finaneial Offieer), rather 
than eonneeted to Provost Offlee (aeademie exeeutive braneh of the institution). Despite the ATP’s baekground in 
edueation only 22%, less than 'A, of the study respondents were eonneeted to the provost offiee. Over % of the AT 
professionals who partieipated in the study are part of a non-aeademie department of the eollege or university. 

ATP’s who are affiliated with the offiee of the Aeademie Exeeutive (Provost Offiee) underperform strategie 
planning responsibilities. 

The strategie importanee of teehnology to the university has inereased and entrusting AT to individuals 
who are experts in teehnology but not in pedagogy is dangerous (Jaekson, 2011). AT is mission eritieal to the 
university and deeisions regarding AT need to be made by individuals who have a strong understanding of the 
aeademie goals of the university (Bates & Sangra, 2011). Aeeording to Sellers (2005), IT is eentral to the 
edueational administration of higher edueation institutions, yet as Brown states, IT departments have, in the past, 
been eonsidered indifferent at best, and openly resistant at worst, to the needs of their eustomers: students and 
faeulty (Brown, 2004). Furthermore, with IT at the eenter of the administrative stmeture, teehnology is driving 
many aeademie deeisions (Sellers, 2005) and there seems to be a laek of eonsensus on how teehnology should be 
used and integrated into universities (MA & Runyun, 2004). The CIO, typieally the head of IT, is eharged by the 
university to ensure that his institution uses teehnology to its maximum long-term benefit (Jaekson, 2011) and to 
many, teehnology for teaehing and learning is a desirable nice-to-have rather than a eore eomponent of the 
institutions teehnology repertoire. Teehnology is best integrated into teaehing and learning when the related 
deeisions are made in eonjunetion with other aeademie deeisions sueh as eontent, pedagogy and teaehing methods 
(Bates & Sangra, 2011). 

The literature emphasizes the importanee of having an AT professional, who understands teehnology and 
it’s pedagogieal uses, in a position of importanee and leadership in the institution (Nworie & Albright, 2008; 
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Privateer, 1999). The data elearly proves that a minority of AT professionals are affiliated with an aeademie 
exeeutive braneh of the institution, sueh as the Provost offiee, rather, the majority of ATP’s are found within the IT 
units of their institution. It is an interesting phenomenon that AT professionals seem to be relegated to teehnology 
positions and not aeademie positions. The majority of the AT professionals have a baekground in edueation, but 
when the term “teehnology” was added to their position, they beeame teehnology professionals. 

AT Professionals Have Opportunity To Share, Do Influenee, But Not Consulted 

AT professionals report being given the opportunity to express an opinion or reeommendation and when 
they do so they believe that their reeommendation has an effeet on the deeision making process. Seventy-four 
percent of the respondents strongly-agree or agree that they are given an opportunity and an additional 14% were 
ambivalent about whether or not they had an opportunity to express a recommendation on an AT decision. More 
than 2 / 3of the respondents believe they have the opportunity to effect the AT decision making process at their 
institution. 

When AT professionals are given the opportunity to express a recommendation 61% believe that their 
recommendation effects the AT decision making process at their institution. A majority of those who do have the 
opportunity to make a recommendation believe that their views are considered as part of the decision making 
process. 

The literature suggests that decisions for integrating technology into teaching and learning should be made 
in conjunction with other academic decisions under the leadership of an AT professional (Bates & Sangra, 2011; 
Nworie & Albright, 2008; Privateer, 1999). Privateer continues to express the need for an institution wide AT 
agenda under strong AT leadership (Privateer, 1999). The data shows that current ATP’s believe that they have 
influence in AT decision making, but as they are overwhelmingly affdiated with IT and not academia their influence 
is limited to the technological side of the institution. 

AT professionals who are skilled in both technology and pedagogy have the ability to influence curriculum 
reform and changes in teaching methods that facilitate the development of skills in a particular subject domain, and 
by influencing changes in assessment ensure those skills are evaluated (Bates & Sangra, 2011). These additional 
responsibilities and successful change requires the opportunity to influence the AT decision making process. The 
data supports this opinion that AT professionals have an opportunity to express their opinion or recommendation 
and influence AT decision making at their institution. 

Summary 

A descriptive study was conducted to describe the current AT professionals perception of their influence on 
the decision-making process at their institution. Seventy-one AT professionals completed an online survey and 
statistical factor analysis of the responses identified the five most cited responsibilities and compared differences 
between what the professionals “do” and what they believed they “should” do. ANOVA was used to calculate the 
resulting gaps in performance based on the “do” and “should” responses. 

The data supported three conclusions: a) AT professionals underperform in their AT roles; b) AT 
decisions appear to be made based on technological rather than pedagogical considerations; and c) when given an 
opportunity to make a recommendation, they influence AT decision making, but they are rarely given the 
opportunity. AT is a fast-growing field that deserves attention, given its dynamic nature and its impact on 
educational practices. The AT field is different from IT, as it concentrates on advancing technology to enhance 
teaching and learning. Yet the field appears to be insufficiently mature or defined, making it difficult to for AT 
professionals to be categorized and characterized or to fully understand their changing roles. 

AT is at the convergence of pedagogy and technology. AT professionals are in the unique position of being 
able to influence technology decisions based on educational goals and assist faculty to use technology appropriately 
to achieve the desired outcome. To ensure student success, higher education institutions need to implement an AT 
plan to support the needs of the faculty and students. The AT plan needs to be implemented by professionals in the 
field who are uniquely qualified to bridge the gap between academia and technology. 
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Abstract 

This presentation will look at technology tools and apps for K-12 education. It will showcase resources and 
also share the results of a teacher survey to differentiate between tools that sound good versus those that work well 
for teachers. Video vignettes and interviews will feature teachers describing these tools in their own words. 
Participants in the presentation are encouraged to follow along with mobile devices so they can download apps and 
access the tools in real time. 


Technology Integration in K-12 

Successful technology integration by K-12 teachers is the subject of ongoing, ever-evolving inquiry. One 
hallmark of teachers who successfully integrate technology is that they possess a “strong sense of personal 
ownership of the (mobile) device and associated activities” (Kearney & Maher, 2013). This research also indicates 
various aspects of the use of mobile technologies and apps in the classroom including: personalization, authenticity, 
and collaboration (Kearney & Maher, 2013). By introducing teachers to technologies that will work in their 
respective classroom settings, these tools will be more likely to improve students’ learning. Specifically, research 
shows that when teachers recognize the impact that technology tools can have on their students’ learning, they are 
more motivated t to continue experimenting with these kinds of tools in the future (Ottenbreit-Leftwich, 2007). 
Additionally, research indicates that teachers’ acquisition of technology knowledge and expertise can lead to 
positive changes in self-efficacy, pedagogical belief, and culture (Ertmer & Ottenbreit-Leftwich, 2010). 

Introduction 

Recent years have brought about an unprecedented proliferation of technology tools designed for K-12 
education. Many of these tools are app-based and either free or relatively inexpensive. These tools have opened up a 
world of possibilities, while at the same time relegating once pricey, stand alone tools like sets of classroom clickers 
to the junk pile. Over 3000 education apps are currently offered through the Apple App Store as are many others 
through Google Play. For teachers who are new to these tools or who wish to expand their repertoire, choosing 
quality tools can be difficult due to the sheer volume of available options. So where is a motivated, forward-thinking 
teacher to start? 

When considering new tools for their classrooms, teachers should first look to their colleagues. These 
colleagues may include not only pre-service and experienced in-service teachers, but also teacher educators, school 
librarians, technology specialists, and other K-12 related service professionals interested in technology integration. 
By consulting colleagues such as these, technology-seeking teachers can better differentiate between tools that 
sound good versus those that are actually working well in real classrooms. 

The technology tools highlighted in this presentation and paper are appropriate for a variety of student 
populations and instructional situations including preschool, elementary, middle/high, special education, and 
students receiving support via the Response to Intervention (Rtl) process. The technology tools will be introduced 
and demonstrated by two presenters whose roots are in K-12. One presenter is a professor of educational technology 
and former high school Biology teacher. The other presenter is both a K-12 school psychologist and educational 
technology doctoral student. 
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To further explore this topie, a brief survey was specifieally developed to ask K-12 edueators what tools 
they eurrently use in they elassrooms and would reeommend to others. Additionally, several other teaehers were 
interviewed and videotaped speaking about their favorite teehnology tool. Tools eovered in this presentation inelude: 
Nearpod, Gaggle, iPadio, Edmodo, Zondle, ClassDojo, FutureMe, ToonCamera, and more. 

The Survey: What’s Working in K12 

At the time of this report, 32 teaehers and other K-12 edueational professionals had responded to a brief 
survey eondueted speeifieally for this paper and presentation. The survey was administered via Google Forms and 
distributed through the soeial networking site Faeebook. Speeifieally, a eonvenienee sampling method was used, and 
individuals who ehose to take the survey were eneouraged to pass the link on to their other K-12 edueator friends. 
The survey eonsisted of the following items: 

• Flow would you deseribe your role? 

• In what state are you eurrently employed? 

• Flow many years of experienee do you have working in edueation? 

• How often do you use teehnology tools (e.g., mobile apps, tablet eomputers) in your elassroom? 

• If you indieated never or oeeasionally, why? 

• If you indieated “yes” (e.g., oeeasionally, weekly, or daily), how do you use these tools? 

• Please name your most used teeh tools (list up to 3). 

• Name any tools that you’re interested in using but haven’t yet tried. 

• If you answered the previous question, what, if anything, is preventing you from using a particular 
tech tool with your students? 

• Have you attended any professional development related to tech integration? 

• Does your school district offer technology-related professional development? 

• Which answer best describes the origins of the tech tools you use with students? Options: a) tools 
that I learned about in school, b) my own tools/devices; or c) tools provided by my district. 

Interesting Findings & Themes 

As previously noted, 32 teachers and other K-12 educational professionals responded to a survey about the 
technologies they use in their classrooms and/or daily practice. Specifically, a breakdown of the responses provided 
by the classroom teachers and their characteristics including position, years of experience, frequency of technology 
use, and perceived barriers to access are reviewed below: 

• High School teachers (2 total) - all came from Kentucky, all had from 5 to 20 years teaching (avg. 10.67 
years), used technology weekly to occasionally (avg. 1.67), all had barriers relating to lack of access, all 
use their own devices when using tech tools with students. 

• Elementary teachers (7 total) - mixed locations, mixed experience (all but one has over 5 years of 
experience - avg. 9.07 years), most used tech daily, but one was never and one was weekly (avg. 2.43), 5 of 
the 7 reported they use technology for planning/organization and instruction. 
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• Middle school (4) - 3 of the 4 were from Kentucky, all had over 5 years experience (avg. 15.25 years), 3 of 
the middle school respondents use tech daily & 1 weekly (avg. 2.75), all used technology for 
planning/organization and instruction, all have attended tech-related PD, all are offered tech-related PD by 
districts. 

• Special Ed. (9) - 2 of the 9 were from Ohio and the other 6 were from KY, all had over 5 years experience 
(avg. 11.67 years), 7 use tech daily and 2 use it weekly (avg. 2.78), 8/9 use for planning/organization and 
8/9 use it for instruction, 8/9 use for games/student rewards. 

The results of this brief survey provide only a snapshot of the current issues impacting teachers’ use of 
technology in the classroom. The presentation, brief survey, and video vignettes shed light on directions for future 
inquiries and ways in which the research can be tightened up and focused. 

Going Forward 

Although research on technology integration in the classroom is prevalent, researchers and practitioners 
should continue to gather data on the topic in order to clarify factors including potential barriers to effective use, 
teacher attitudes toward technology, and avenues for technology-focused professional development. Whether it be to 
deliver instruction, facilitate planning/organization, or to use as incentives/rewards for students, technology 
integration and the potential it holds for the classroom is at the forefront of many teachers’ minds. As educational 
technology professionals, our ability to link research to practice and professional development may be the greatest 
gift our field can give to K-12 education. 
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Abstract: Research on the efficacy of diagramming complex arguments has been mixed. One 
reason for the mixed findings is that the precise processes students use to construct argument 
diagrams have yet to be fully examined. This study identified sequential patterns in argument 
diagramming actions performed by graduate-level students that created high versus low quality 
diagrams. Transitional state diagrams revealed patterns in the action sequences used by high vs. 
low performing students - processes believed to either help or inhibit students’ ability to construct 
accurate argument diagrams and achieve a better understanding of complex arguments. The 
findings reveal processes that can be embedded into future diagramming software to test how 
particular processes affect students’ analysis and understanding of complex arguments. 

Purpose 

Research on the efficacy of using visual diagramming tools to facilitate argument analysis has been mixed (Braak et 
al., 2006; Ruiz-Primo & Shavelson, 1996). One reason for the mixed findings is that empirical studies have yet to be 
conducted to formally identily the sequential steps and reasoning processes students use when constructing 
argument diagrams. This study developed a set of visual analytic software tools to record, sequentially analyze, 
visualize, identify and compare sequential patterns in argument diagramming actions performed by graduate -level 
students that created high and low quality argument diagrams. Transitional state diagrams were then created and 
used to visualize sequential patterns found in the actions of students that created high and low quality argument 
diagrams. The transitional state diagrams were then compared to reveal action sequences that were used by high 
performing students, but not used by low performing students and vice versa. The unique action sequences 
performed only by the high performing students can reveal the types of diagramming processes that can be promoted 
and scaffolded to help students construct more accurate argument diagrams and improve understanding of complex 
arguments. 


Introduction 

Critical thinking is an important skill that enables one to accurately reason and judge information and become 
lifelong learners for the 2P‘ century. It has been defined as ‘the art of analyzing and evaluating thinking with a view 
to improving it’ (Paul & Elder, 2001) and an intellectual standard that includes clarity, accuracy, precision, 
relevance, depth, logic, and breadth (Mclean, 2005). However, recent research suggests that many college students 
fail to develop critical thinking skills to the extent that they can effectively use them (Kuhn, 1991). To address this 
problem, various methods have been used to teach students the skills of argumentation and argument analysis across 
many disciplines. Argument analysis is the study of logical relationships between propositions presented in an 
argument (which can be mutually supporting or opposing opinions/claims) in order to reason through premises to 
reach a conclusion. In argument analysis, students identify the functional roles of each proposition (i.e., conclusion, 
premise, co-premise, counterargument), analyze the hierarchical relationship among propositions (i.e., levels of 
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premise), and evaluate the quality and line of reasoning. This process helps students to correctly judge the quality 
and identify flaws within an argument and help students to make well-reasoned decisions. 

Given that arguments are often complex and ill-structured, argument-mapping software like Belvedere (De 
Neys, 2006) and Rationale (van Gelder, 2007) have been developed to help students visualize/identify hierarchical 
relationships between minor/major premises and claims (Braak, 2006). Some diagramming software, like REASON 
(ThinkReliability, 2007), prescribe the use of specific logic rules and processes such as the backward reasoning or 
goal-driven approach (Sharma, 2012). Yet, a critical review of the research on argument diagramming/visualizing 
tools revealed that the majority of the studies found no significant differences (Braak at ah, 2006) and/or were 
flawed in design. Furthermore, students’ maps often varied widely in accuracy regardless of the instructional 
intervention (Scavarda et ah, 2006). Ruiz-Primo and Shavelson’s (1996) review of the research lead to the 
conclusion that students’ diagrams should not be used to assess learning until students’ facility, prior knowledge, 
and processes used to create the diagrams are thoroughly examined. 

Given that no studies at this time have modeled, identified, and/or validated prescribed mapping processes that 
enable/inhibit students’ to accurately analyze complex arguments. Author (2010) created j MAP to chronologically 
log each action a student performs while constructing an argument diagram in jMAP. This data can be sequentially 
analyzed to identify the processes used by students to produce high and low quality argument diagrams. In 
particular, processes associated with informal reasoning fallacies performed while constructing argument diagrams 
can in theory be detected by observing diagramming processes. For example, a hasty generalization or leap to 
conclusion can be observed when a student creates a link (D^A) when the effects of D on A is mediated by B 
(D^B^A). Circular reasoning can be observed when a student links A^B and B^A. To date, no studies have 
examined the diagramming/actions sequences associated with reasoning fallacies (including actions that 
immediately precede/follow such action sequences) in the course of constructing argument diagrams. Using this 
approach, this study addressed the following questions: 

1 . What sequential patterns in students’ diagramming actions produce the most versus least accurate argument 
diagrams? 

2. What are the differences in processes used to produce the most versus least accurate argument diagrams? 

Method 

Seventeen graduate students in an online graduate-level course on computer-supported collaborative learning at a 
large Southeastern university reviewed arguments produced by students in an online debate (but from another 
course) to support/oppose the claim: “One’s choice of media significantly affects learning”. After viewing a video 
on how to use jMAP, students downloaded a jMAP file to diagram the supporting arguments and another jMAP file 
to diagram opposing arguments. In both cases, students were presented an initial screen (Figure 1) containing nodes 
that represented the main claim and supporting/opposing premises. 
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Figure 1 . Students’ initial screen with premises randomly arranged on the screen. 

The students’ diagrams were imported into a jMAP fde containing the instructors’ argument diagram (Figure 2) 
to score the students’ diagrams on three criteria presented in Figure 3: a) percentage of links that match those in the 
instructor’s map; b) number of nodes correctly identified as a root premise (node with no arrows pointing inward); 
and c) number of correct links stemming directly from each correctly identified root premise up to the main claim. 
Using a cumulative score to identify diagrams with high versus low accuracy, data from the top 6 and bottom 6 
diagrams of the supporting arguments and from the top 6 and bottom 6 diagrams of the opposing arguments were 
selected to produce 12 best and 12 worst diagrams. The logged actions (Figure 4) recorded in jMAP from the bottom 
diagrams were aggregated and then reduced down into six categories to capture more general patterns in students’ 
actions. The same process was repeated with the top diagrams. 

The logged actions from the bottom diagrams were imported into the Discussion Analysis Tool (DAT) to 
produce the frequency, transitional probability, andz-score matrices (Figure 5). The frequency matrix shows for 
example that when these students added a link, 40 of the 129 actions that immediately followed were to add yet 
another link. To determine if this transitional probability of .3 1 was significantly higher/lower than expected 
probabilities (and whether AddLink^AddLink can be deemed to be a sequential “pattern”), the z-score matrix 
shows for this particular action-action sequence a z-score of 6.55 (which is greater than the critical z-score of ±1 .96 
aXp< .05). As a result, the AddLink^AddLink sequence was found to be a sequential pattern in the actions used to 
produce the bottom diagrams. DAT converted the probabilities into the right transitional state diagram (Figure 6). 
This process was repeated with data from the top diagrams to produce the left transitional state diagram. 
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Note: Dark/gray colored arrows identify links present/missing in student x’s diagram; Nodes with green halos 
identify lowest level premises correctly identified by student x. 


Figure 2 . Instructor’s diagram visually and quantitatively compared with student x’s diagram. 
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Figure 3 . Screen shot from jMAP showing scores assigned to students’ diagrams of supporting arguments. 
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Figure 4 . 


Codes assigned to each action students perform in jMAP while constructing an argument diagram. 
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Note: The values identified in bold/underline identify action sequences occurring at higher/lower than expected 
frequency based on the critical z-score of ±1 .96 atp< .05. 

Figure 5 . Screen shot from DAT showing the frequency, transitional probability, and z-score matrices used to reveal 
sequential patterns in the actions used to create the 12 bottom diagrams. 

Main Findings 

The two state diagrams (Figure 6) reveal four action-action sequences that were common in both groups 
(AddLink^AddLink, MoveNode^MoveNode, and DeleteLink^DeleteLink, AddLink^ ChangeLinkAttribute). 
The first three action sequences suggest for example that students constructed their diagrams using a stage-like 
sequence by moving multiple premises into position first, then inserting links to connect the premises with links, and 
then deleting (or correcting) the links between premises. Given that these four processes were observed in both 
groups, the findings suggest that the use of these four processes neither increases nor decreases the accuracy of 
students’ argument diagrams. 

The diagrams also show that top scorers exhibited five unique action sequence patterns. For example, the 
left diagram shows that when top scorers deleted a link, they were most likely (44%) to follow that action by adding 
a new link between nodes than bottom scorers. When they specified the attribute of the link, they were most likely 
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(35%) to follow that action by adding a new link. Overall, the differences between the two state diagrams suggest 
that the following five action sequences (when performed on a consistent basis) can help students construct more 
accurate diagrams: DeleteLink^AddLink, Attribute^AddLink, Attribute-^ Attribute, Relink Effect (move the head 
of the arrow to point to another affected node)-> Relink Effect, and Relink Effect^Relink Cause (move tail of arrow 
to point to another causal node). In particular, the sequence of DeleteLink^AddLink may be an indication of times 
when students are restructuring their diagrams to undo an error produced when making hasty generalizations (when 
A->Conclusion and B^Conclusion should in fact be restructured to A^B->Conclusion). 




RELINK 

CAUSE 

11>11 


MOVE 

836>827 


DELETE 

LINK 

10>10 


ATTRIBUTE 

31>30 


Bottom 12 Diagrams 



Note'. Thickness of arrow conveys strength of transitional probability; dark black arrows identify probabilities that 
are significantly greater than expected based on z-score tests (p < .01) performed in the DAT software; first and 
second numerical value displayed in nodes identify the number of times the given action was performed and the 
number of events that followed the given action; the size of the glow emanating from each node conveys the number 
of times the action was performed. 

Figure 6 . State diagrams of processes used to produce the top vs. bottom diagrams. 

In contrast, the low-performing students exhibited four unique action sequence patterns (AddLink 
^RelinkCause (change tail of link to point to a supporting or subordinate premise), AddLink^RelinkEffect 
(change head of link to point to a superordinate premise). Relink Cause^Relink Cause, and Relink 
Cause^ChangeLinkAttribute. Given that low scorers exhibited the tendency to perform the 
RelinkCause^RelinkCause sequence whereas high scorers exhibited the tendency to perform the 
RelinkEffect^RelinkEffect sequence, these differences suggest that the processes used to: a) create more accurate 
argument diagrams involved the use of a forward or bottom up approach - systematically examining which major 
premise (or effect) is supported by a given minor premise (or cause); and b) create the less accurate diagrams 
involved the use of a backward or top-down approach by progressively examining which minor premise supports a 
given major or superordinate premise. 

To determine if the differences in the observed patterns are statistically significant, a phi-coefficient (o) 
will be used to measure the strength of association between group membership and particular pattern (Bakeman & 
Gottman, 1997). Retroactively recoding and further analysis of the action logs will examine actions (and the actions 
preceding them) that produced correct versus incorrect links across the argument diagrams of all 17 students. 

Conclusions and Implications 

Although the findings are not conclusive, the observed differences in action sequences used by students to construct 
argument diagrams with high versus low accuracy provide initial insights into the types of processes that can help 
students create more accurate and achieve deeper understanding of complex arguments. At minimum, the findings 
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and the possible interpretations of the meaning behind the findings provides some insights into the possible methods 
and approaches that can be used in future research to model and better understand the processes that can be used 
effectively to analyze and better understand complex arguments. 

Future work is needed to: a) replicate this study with larger sample sizes; b) apply multiple approaches to 
establish validity in the criterion used to assess diagram accuracy, c) refine the precision of the data mining codes in 
jMAP to fully determine if students are in fact linking premises using a forward or backward approach and are 
sequentially relinking premises into logical chains to correct for errors produced by making hasty generalizations; d) 
identify which diagramming actions reflect general reasoning processes that improve argument analysis (areas B and 
E in Figure 7, respectively); e) integrate the target action sequences directly into the software interface of the 
diagramming software so that controlled experiments can be conducted to determine cause-effect relationships 
between target processes and map accuracy; f) determine to what extent particular processes are dependent on 
students’ prior knowledge in order to identify the target processes that can be promoted regardless of students’ prior 
knowledge; and g) examine to what extent the target processes are effective across arguments varying in hierarchical 
structure/complexity. 



Figure 7 . Areas for further research on the relationships between diagramming processes, general reasoning 
processes, and understanding of complex arguments. 
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This is a review of published research on using blogs for collaborative online learning. One aim of the 
review is to explore the impact of collaborative learning activities on students using group blogs for collaboration 
processes. A second purpose is to review learning outcomes from collaborative blogging reported in research. 
Blogs have some attributes that can potentially facilitate collaboration. Because collaboration is important, we 
should seek to understand how online tools such as blogs can be used to foster collaborative activities and how 
they can be used to facilitate learning The factors for collaboration are grouped into four segments: 1) importance 
of collaboration; 2) blogs as a collaborative tool; 3) role of motivation when collaborating via blogs; and 4) 
evaluating learning when collaborating via blogs. 


Introduction 

The use of online tools has become an everyday affair for everything from social correspondence, to 
learning, to conducting business. Blogs (online time-stamped journals) are widely used by organizations, groups, 
and individuals to voice opinions and to share information and receive feedback from their readership. Research 
shows blogs have been commonly used in online learning environments for collaboration for over a decade (Avci & 
Askar, 2012; Boulos & Maramba, 2006; Bouwma-Gearhart & Bess, 2012; Byington 2011; Chen, Cannon, Gabrio, 
Leifer, Toye & Bailey 2005; Ahrens & Zascerinska, 2010). Considering that more than 6.7 million students took at 
least one online course in 2012 (Allean & Seaman, 2013), a review of the use of blogs for collaboration in online 
learning is timely. 

One aim of the review is to explore the impact of collaborative learning activities on students using group 
blogs for collaboration processes. A second purpose is to review learning outcomes from collaborative blogging 
reported in research. Blogs have some attributes that can potentially facilitate collaboration. Because collaboration 
is important, we should seek to understand how online tools such as blogs can be used to foster collaborative 
activities and how they can be used to facilitate learning (Avci & Askar, 2012; Boulos & Maramba, 2006; 
Bouwma-Gearhart & Bess, 2012; Byington, 2011). This review aims to address the following questions: 

• How collaborative is the blog based learning technologies for students and to what extent do students 
believe that blogs can enhance collaboration? How consistent is blogging to the learning design method? Can blogs 
efficiently facilitate the communicational and information management requirements? 

• To what extent do the blog based collaborative interactions affect self-regulation of the group blog 
members? Are there any significant differences for impact among the different kinds of blogs? What about their 
collaborative learning outcomes? Is it similar or different to class collaborative work? 


Importance of Collaboration 

Collaboration is a well-known instructional strategy associated with knowledge development. 
Collaboration encourages reasoning and critical thinking. Laughlin, Hatch, Silver and Boh (2006) found that groups 
of three or more persons perform better than equally intelligent individuals in a group of one or two when solving 
complex problems. Furthermore, researchers found that collaboration helps develop reasoning skills and abilities to 
solve complex problems and acts as a source of knowledge development (Slavin, 1995; Sorensen &Tackle, 2002; 
Wheeler et al, 2008). Other researchers found that with successful collaboration participants further develop skills, 
such as: critical thinking, reflective thinking, providing constructive feedback, and teamwork (Chandra & Chalmers, 
2010 ). 
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Several faetors have been shown to impaet the effeetiveness of eollaboration (e.g., time management, 
synehronization, and aetive eommunieation) (Coleman, Austin, Braeh & Wagner, 2009). Division of labor 
(distributing work to members) has also been found to be a key influenee on the sueeessful implementation of 
eollaborative learning (Bouwma-Gearhart, & Bess, 2012). In most situations, eollaboration requires individuals to 
work together as a team to solve problems. If one of these aforementioned eonditions is missing or implemented 
poorly, eonfbets may arise that negatively affeet knowledge development (Ishtaiwa & Abulibdeh, 2012; Kerawalla, 
Minoeh, Kirkup & Coneole, 2009). However, when implementation of eollaboration is based on strategies shown 
as sueeessful, the growth of knowledge is eneouraged through eonstruetive feedbaek from partieipants (Chandra & 
Chalmers, 2010; Yang, 2009). Therefore, to properly understand what happens during eollaboration, we need to 
understand the eollaboration proeess. 

Collaboration Process 

Collaboration ean be defined as a proeess where two or more students join learning resourees and 
individual knowledge to aehieve eommon learning goal. Curtis and Lawson (2001) identified the following 
behaviors as a supportive form of eollaboration: (a) giving and reeeiving help and assistanee, (b) exehanging 
resourees and information, (e) explaining or elaborating information, (d) sharing knowledge with others, (e) giving 
and reeeiving feedbaek, (1) ehallenging others eontributions, (g) advoeating inereased effort and perseveranee 
among peers, and (h) monitoring eaeh other’s’ efforts and eontributions. Aeeording to Arsenyan and Buyukozkan 
(2009), a eollaboration proeess has four dimensions: trust, eoordination, eoleaming, and eo-innovation. 

Building open trust-based relationships is the key to sueeessful eollaboration. Aeeordingly, eollaboration 
ean be aehieved by working through three stages: exehange of vision, negotiate agreement, and negotiate tmst 
(Greenfield, 2003). The eollaborative team needs to look for trusting members to rely and to eoordinate sueeessfully. 
Coordination is another key faetor in eollaboration partieularly if teams from different organizations and loeations 
are engaged for the same purpose. Improved learning is partly the result of effeetive eommunieation and information 
distribution systems both within and between organizations (Carayol & Roux, 2005). Coleaming emerges as another 
key issue in eollaboration. Identifying relevant knowledge inputs from the various partner organizations needs to be 
viewed as a eonstant (Arsenyan & Buyukozkan, 2003). 

Information sharing emerges as an important aspeet of eollaboration. A key goal for blogbased 
eollaborative learning is to shift from an essentially statie approaeh of learning that is based on information 
aequisition towards a greater emphasis on information interpretation and distribution. Use of learning teehnologies 
are ehanging rapidly whieh drives the need for inereasing information sharing and eooperation. Knowledge is 
aeeumulated both through internal eapaeities of the agent and through the direet and indireet eonneetions that allow 
aeeess to others' knowledge (Carayol & Roux, 2005). 

Blogs as Collaborative Tools 

The word 'blog' is a short form of weblog. When effeetively utilized, blogs ean offer a way to enhanee 
students' learning experienees and deepen levels of learners' engagement and eollaboration within digital learning 
environments. Therefore, researeh should be eondueted to determine the best ways to integrate blogs into existing 
eollaborative e-leaming programs for student learning. Advantages of using blogs in online learning inelude ease of 
use, low eost (free 

- e.g., Google blogs), and broadeasting. The disadvantage is the quality of the blog eontent and the laek of eontrol 
over the eontent. Monitoring and moderating blog posts, deleting as neeessary, rendering red-only option , 
eonfrolling who ean post, and bloeking speeifie user (IP) are some possible remedies in blogs quality 
management. 

Early on, Tailor-Powell et al. (1998) suggested that for online eollaboration to be effeetive, partieipating 
members must praetiee eommunieation, eontribution, eoordination, and eooperation. Subsequent studies reveal that 
managing and produeing eontent via a blog ean improve a student’s skill in writing, responding, eommunieative 
skills, and synthesis of information (Ishtaiwa & Abulibdeh, 2012; Robert & Dennis, 2005; Tekinarslan, 2008). 

Curtis and Lawson (2001) found that blogs were a more effeetive eollaborative environment for peer learning than 
other tools sueh as wikis. Thus, it ean be said that blogs favor exehange of ideas through posting new eomments 
and replying to other eomments mueh easier instead overwriting a previous eomment/post. 

As blogs engage people in knowledge sharing, refleetion, and debate, they often attraet a large and 
dedieated readership (Lederer, Greenberg, Muehlen & Ralph, 2003). Henee, blogs ean easily draw like-minded 
people together in a small form of virtual groupings where individuals interested in eo-eonstrueting knowledge 
around a eommon topie within a eommunity of praetiee ean easily eollaborate together. 
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Wu, (2006) in Taiwan reported on blog implementation for peer review and teacher feedback in an 
Intermediate EFL (English as a foreign Language) composition class that was composed of adult university 
students (n = 39). The course lasted for 18 weeks and combined course meetings with two peer reviewed blog 
writing sessions. The analysis of the data showed that most of the students were easily encouraged and motivated 
by their peer feedback and constructive critism due to the fact that all students were to carry out a peer review 
during the course. 

In another study, the teacher’s feedback and Ferris’ subjective rating scale with values ranging from 0 to 6 
points (1997), were applied to two different drafts submitted before and after a blog peer review. The results showed 
little progress in students’ writing skills. Murray and Flourigan’s (2008) Irish study took a step toward articulating 
the special nature of a blog. Blogging was required in an academic writing class composed of students majoring in 
various modem languages (n = 42). In their assumption, the blog environment serves two different kinds of 
activities, one expressive (creative and reflective) and the other socio-cognitive (critical and collective). The students 
reflected on their language learning process through blogs in their target language in which the students were 
majoring. Flowever, students were required to write a 3000word essay in English to integrate the blog entries. 
Although they concluded that the use of a blog had a positive role in self-reflection, the roles of the two investigated 
activities were difficult to discern. 

Wang (2009) tested blog-based electronic feedback (e-feedback) with EFL (English as a foreign 
Language)Taiwanese university students in Taiwan (n =30). The course consisted of a ten-week writing class and a 
four- week blogging project. For the four week blogging, the blog system was used for pre-/post essay writing and 
peer feedback. In their comments, the students were allowed either to identify themselves or to remain anonymous. 
The submissions were corated in terms of their linguistic function and accuracy and were evaluated statistically. 
Thus, the students’ attention tends to be uneven. In other words, the students were focused more on the micro-level 
(lexical and grammatical) or on lower - order concerns (LOCs) and less on the macro-level (organization and 
content) or on higher-order concerns (FIOCs). 

A blog is an interactive collaborative tool, where editor/writer publishes and collects personal information 
according to the purposes and plans of the blog editor (Makri & kynigos, 2007). Blogs generally include the 
following five features: (1) content which is separated from presentation, (2) presentation templates which are 
usually provided for bloggers to easily produce blog articles, (3) blogging application programming interfaces 
(APIs) also help bloggers to use software, such as Microsoft Word, to publish content to blogs, (4) many 
information management tools which are offered in blogging systems for content management, and (5)“Really 
Simple Syndication” (RSS) a mechanism which offers a subscription to the blogging clients who are interested in 
specific topics (Lindahl & Blount, 2003). 

Blogs have the ability to support informal communication, shared learning, group reflection, and 
community building (Nardi et ah, 2004; Divitini et ah, 2005). Due to these characteristics of blogs, it has begun to 
be used in educational settings in recent years. An implemented blogging system has been proven to be feasible in 
an international distance course (Lin et ah, 2006). Students were able to use the system to document their learning 
and to share experiences and knowledge. For the moment, blogs are able to provide valuable materials for 
supporting students’ learning (Fluang et al., 2008). In addition, blog articles were functional to construct a learning 
map called blog-based dynamic learning map (Wang et ah, 2008). It is designed to provide informative blog articles 
to assist students’ learning. Furthermore, difficulties around the issues of direct student-student and student-tutor 
interactions were addressed. Flowever, some delivery features of a blog increased anxiety among some students 
resulting in the inefficient use of learning time (Dron, 2003). Through the application of different structural and 
methodological techniques, these problems can be overcome. Using the learning research methods and learners’ 
experience presented in a blog (Giarr'e & Jaccheri, 2006), researchers or learners have been able to study the 
relationship between the research process and results (Giarr'e & Jaccheri, 2005). 

In blogs, participants can discuss research problems, exchange information by drawing upon the 
knowledge of each other, collaboratively find solutions to their research problems, and find/discuss new 
applications for concepts without any constraints in terms of time (Mai, 2005; Nicola & Giuseppe, 2006). 
Nevertheless, most of the blogging actions take place on PCs or laptops which causes the temporal and spatial 
limitation of publishing blog entries. Under these constraints, blogging could make students unaware of the features 
and the problems encountered in authentic context. Huang, Jeng and Huang (2012) studied an educational mobile 
blogging system which enables mobile bloggers to publish their comments in authentic context anytime and 
anywhere. They showed that with the help of the mobile blogging system that students can establish a collaborative 
learning with positive outcome in virtual classrooms seating. Thus, the mobile blogging system can provide more 
authentic context learning examples and help in solving the coordination issue in a collaborative learning 
environment. 
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In addition, the developed mobile blogging system allows students a similar manipulation of web-based 
blogging system in daily life with no position and time limitations. The result of the study implies that the mobile 
blogging system can be used for providing authentic context of learning example. For instance, students can share a 
picture of a train which can be an example of Linked Lists concept and upload it to mobile blog. After students read 
the blog article, they can discuss and comment on it. One student can illustrate the concept of the figure, and the 
others can look for different example more appropriate. By that manner, students learn to coordinate their efforts 
toward consistent and correct outcome and elaborate the learning material in a collaborative learning setting. 

Role of Motivation When Collaborating via Blogs 

Motivation is a major component of learning (Anderman & Bawson, 2011). Motivation involves both 
social and cognitive factors for learning. There are two primary types of motivation -intrinsic and extrinsic (Ryan & 
Deci, 2000). Intrinsic motivation is the natural inclination to learn and assimilate or do something without the need 
of external push. Blogging can generate intrinsic motivation in that students can view and compare each other’s 
work (Chandra & Chalmers, 2010). Intrinsic motivation can potentially lead to high-quality learning and creativity, 
so it is important to identify factors that can foster intrinsic motivation (Ryan & Deci, 2000). For example, positive 
feedback increases intrinsic motivation (Deci, 1971; Elliot & Harackiewicz, 1994), and negative feedback reduces 
intrinsic motivation (Deci & Cascio, 1972). Because some activities may not be intrinsically motivating to learners, 
educators may employ extrinsic motivators. Extrinsic motivation occurs when an activity is done or needs to be 
done as a result of external interest and not form within (internally). Social tools, such as blogs can be a significant 
motivational tool in learning, teaching, and assessment (Ajjan & Hartshome 2008; Avci & Askar, 2012). An 
instructor can assess a student’s blog to determine the quality and level of interaction and participation by the 
student. The assessment will act as an extrinsic motivator 
for the student. 

Moreover, in the field of collaborative learning with blogs, the collaboration process, motivation of 
learning and positive learning outcomes are of great importance. Collaborative learning activates aims to foster 
productive interactions among the users (students, teachers, researchers) by increasing their awareness and 
facilitating their self-regulation. Besides that, blogs are social software for asynchronous communication, content 
management systems, and recently quite popular collaborative learning environments. It is therefore useful to 
study collaborative interactions for group blogs through which students implement learning activities according to 
collaboration scripts. 

Standard blog features include easy posting, archives of previous posts, and a standalone Web page for 
each post to the blog with a unique URL. The latter feature facilitates linking to and organizing content within the 
same blog and from external sites (Byington, 2011). Posting a clinical photo from a digital camera directly to a 
blog after optimization and adding of a blogger's comments can also be made at the touch of a button using free 
Google product (e.g. Picasa). Moreover, the currently available 3G generation of mobile phones equipped with 2-1- 
megapixel cameras can instantly post high resolution clinical photos to photoblogs/moblogs (mobile blogs) to a 
potentially worldwide audience on the Web. Agosto and Copeland (2013) investigated that the use of blogging was 
effective to encourage student collaboration and knowledge sharing in the face-to-face course environment. 

Medical blog examples include Clinical Cases and Images. Parkes, Dredger and Hicks (2013) evaluate the many 
reflective tasks required to help students increase their level of critical thinking (e.g., in the blogs that they 
contribute to for the field class). 

Instructional design research can find ways in which to innovate and at the same time place innovation 
within the context of sustainable development through collaborative online platforms (e.g., blogs and wikis). 
Therefore, co-innovation appears to be another key factor in collaboration. The search for agreement on the 
innovation applies to both internal and external students as knowledge is accumulated both internally and 
externally. Carayol and Roux (2005) state that expected number of innovation is a function of accumulated 
knowledge. This constitutes a link between co-leaming and co-innovation which cannot be stated separately from 
trust and coordination in collaboration. 

Evaluating Learning When Collaborating via Blogs 

The use of standardized measures to assess collaborative knowledge growth via journaling tools such as 
blogs is not commonly practiced. However, a pedagogical approach known as “folio-thinking” is beginning to 
emerge as a possible approach to evaluation components of collaborative learning with blogs. This practice is 
adopted from e-portfolios using a benchmark tool similar to the one developed by Semar (2005) to evaluate 
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collaborative learning. The “folio-thinking” concept works on the reflective practice of creating portfolios that 
enable students to document and track their learning growth and develop an integrated, coherent picture of their 
learning experiences (Stanford Center for Innovations in Learning, 2002). Using this tool enhances a student’s 
self-understanding of what he or she is learning (Brown, Peterson, Wilson, & Ptaszynski, 2008). Students who blog 
within a course can use “folio thinking” to document their reflective thinking and demonstrate how they developed 
their knowledge. 

As mentioned, the Semar’s benchmark tool is used to evaluate the effectiveness of collaborative learning 
(Semar, 2005). Four important factors of collaboration will be measured using this tool, and they are level of level of 
synthesis, level of freedom, level of interaction and level of participation. These scores for an individual are then 
compared with the rest of the group members. This benchmark is a good tool to measure the level of group 
collaboration and how successful the participants were in solving a given problem. This tool can be adapted to 
assess the level of collaboration (via blogs) of a given group. 

Researchers have also found that successful online collaborative discussion is directly linked to its 
assessment. Simply put, this means that to encourage collaborative discussion one must grade it. Participation for 
discussion must count for a significant portion of the course grade and individual discussion postings must be 
individually assessed. A requirement of a particular number of discussion postings per week or per course module 
will help ensure students participate in discussion. Recent research by Ho (2004) links the ratings on four mbrics 
(quality, quantity, relevance and manner) derived from Grice’s cooperative principles for effective faceto-face 
discourse (Hawisher & Pemberton, 1997) to both the numbers of responses generated by an individual posting and 
the average number of responses generated by particular students suggest that this rubric may elicit collaboration. It 
is certainly worth further investigation. In addition. Ho found that students’ overall Gricean ratings were also linked 
to their final course grades, suggesting the value of collaboration. 

Most importantly Dwyer ‘s (2011), study builds on a new metric for measuring collaborative value from 
the information content of participant contributions to propose a measure of collaborative efficiency and 
demonstrates its utility by assessing collaboration around a sample of weblogs. Dwyer’s research on a 
methodological identification of collaborative performance measure is more broadly applicable approach by being 
algorithmic and therefore automatable and lowers the barriers to taking advantage of the large volume of online 
collaborations. 


Conclusion 

The goal of this review was to investigate how blogs may be used as a collaborative tool and how to 
evaluate learning via blog-based collaboration. As online instraction has become a mainstream, students use of 
tools such as blogs has also become routine. Furthermore, this review examined the latest international studies on 
collaborative learning blogs. We selected some recent quality papers published in peer-reviewed journals that 
reported experimental research on collaborative blogs in various settings. Based on our review, we found the 
following interesting key points: 

1 . Blog based collaborative learning are rapidly growing in instructional technology that needs proper guidelines 
to support quality education, social interaction, knowledge sharing, and motivation of learning (Hsu, Ching & 
Grabowski, in press). Future research should investigate how to promote collaboration and interaction through 
mobile versions of Web 2.0 applications (eg, WordPress, Twitter, Google-i-). 

2. Expanding the context to blog contents: we mostly focused on the issues related to collaboration in blog based 
online learning. Incorporating the verities of learning content (text, images and videos), there is a need to study 
the impact of learners’ cognitive load in blog based learning technologies. Also, we need to analyze the 
confluence of individual cognitive load over collaborative and individual performance. The use of mobile 
technologies might play significant roles and new dimension of future research. 

3. The analysis of qualitative and quantitative assessments of collaboration might fit in blog based instructions. 
There is a need to update the research base by examining the affordances of current mobile technologies as the 
advancement in technologies can afford social interaction and dynamics not possible with earlier technologies. 

Such tools could be of great value and advantageous for students to use for academic collaboration and 
future experiential research. With better understanding of how to integrate social tools and evaluate learning in a 
collaborative environment, we will be able to prescribe effective practices and uses for online social and 
collaborative tools. We hope that our review of studies provides informative analyses and discussions for the 
research community. We also look forward to seeing more quality experimental research in these relatively 
uncharted and promising areas. 
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Introduction 

As technology advances, schools should also react and change to incorporate technology appropriately. In 
fact, in the last 20 years, technology, especially Web 2.0 tools, has dramatically affected how people communicate 
and learn (Solomon & Schrum, 2007). Technology has given teachers more opportunities to design more engaging 
learning environments that help students succeed. Studies have supported the positive effects of technology on 
student engagement and learning, such as searching for information easily, promoting self-expression and creativity, 
and constructing deep knowledge through collaboration and sharing with others (Erickson, 2010; Johnson, 2011; 
Thill, 2011; Yoon & Wang, 2014). 

Erickson (2010) examined the use of blogs as a tool for improving open-response writing in the secondary 
science classes compared to handwritten dialogue journals. Four classes were equally divided into an experimental 
group using the blog and a traditional group using the traditional journal (Erickson, 2010). Results indicated that the 
blog group had a significantly more positive attitude about the experience than the dialogue journal group. Students 
indicated that that blogging was fun and helpful and made them look forward to science class (Erickson, 2010). 

Initiatives of Using Technology in K-I2 Edncation in Sandi Arabia 

Technology uses in education has been expanded in the last three decades in Saudi Arabia. Starting in the 
early 1990s, educational technology uses, especially computers, were limited to administrative purposes, lesson 
planning, writing assignments and repots, and other classroom management activities (Alshumaim & Alhassan, 
n.d.). To support the spread of computer literacy among the new generation, the Ministry of Education established 
many computer clubs in several cities (Al-Mezher, 2006). In 1999-2000 school year, the Ministry of Education 
decided to change school libraries into educational learning centers that were connected to the internet and equipped 
with computers, projectors, and other multimedia (Al-Mezher, 2006). Later, computer literacy programs as a 
compulsory subject in the secondary school curriculum were introduced where schools were gradually equipped 
with a computer lab and teachers were trained (Al-Mezher, 2006; Alshumaim & Alhassan, n.d.). 

In 2010, the Ministry of Education and King Abdullah bin Abdulaziz Public Education Development 
Project (Tatweer) signed a contract with the Microsoft worldwide program - “Partner in Learning”, which aimed to 
support the ministry and Tatweer efforts to develop education through ICT integration. This partnership focused on 
training policy makers, school leaders, and teachers to gain knowledge and skills in integrating Information and 
Communication Technology in the learning process. The training focused on several tools including Microsoft 
software, like Microsoft office, Microsoft publisher, Microsoft Auto Collage, Live Sky Drive, Bing Search, 
Microsoft Mathematics 4.0, Microsoft Movie Maker, Microsoft Photosynth, and Microsoft OneNote. 

King Abdullah bin Abdulaziz Education Development Project (Tatweer) 

In reaction to the increasing criticism to the Saudi curricula and continues calls from stakeholders to 
improve the whole educational system in the country, the Saudi Council of Ministers launched King Abdullah bin 
Abdulaziz Public Education Development Project (Tatweer) at the beginning of 2007. Tatweer is an Arabic term, 
simply means reform. Unlike the previous reform initiatives, Tatweer adopts a comprehensive systemic change in 
the Saudi education system. In addition to curriculum development, others educational aspects are addressed, 
including developing educational standards and assessment to fit the 2P‘ century needs, improving professional 
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development, and enhancing school environment to promote learning (Hakami, 2010). In general, Schools are 
considered as the building block for reforming the Saudi education in Tatweer project. 

Tatweer schools new curriculum emphasizes using new educational technologies to support students’ 
collaboration work with community involvement to help them possess 2P‘ century skills (Tatweer, n.d.). As 
technology is one of the building blocks in this new school system and to help the schools move forward standing on 
a concrete base, it is important to investigate the competency of Tatweer school teachers in using technology based 
on the International Society for Technology in Education (ISTE) National Educational Technology Standards for 
Teachers (NETS.T) widely accepted technology standards, especially neither formal nor informal study has been 
conducted to examine the use of educational technology in Saudi Arabia in relation to these standard. 



Figure 1: Tatweer School 


International Society for Technology in Edncation National Edncational Technology Standards 

The International Society for Technology in Education (ISTE) is the premier membership organization for 
educators and educational leaders (About ISTE, 2011). ISTE promotes professional development, innovation, and 
advancing the effective use of technology PK-12. More than 100,000 members come from across the globe. ISTE is 
the home of the National Educational Technology Standards (NETS), the Center for Applied Research in 
Educational Technology (CARET), and the National Educational Computing Conference (NECC). 

National Educational Technology Standards (NETS) have served as a guide lines since 1998 for improved 
learning and teaching through the proper technology integration (Standards for global learning in the digital age, 
2011). NETS have been widely adopted by U.S. educators and increasingly advocated in countries worldwide. 
Aiming to integrate technology across all curricula, NETS are used to help technology planning and curriculum 
development across primary and secondary school settings. ISTE recently led an international project involving 
thousands of educators and education leaders to update the NETS. The project resulted in updated standards: 

• National Educational Technology Standards for Students (NETS.S): The skills and knowledge students 
need to learn effectively and live productively in a digital world (NEST for students, 2007). 

• National Educational Technology Standards for Teachers (NETS.T): The skills and knowledge educators 
need to change the way they teach, the way they work, and the way they learn in an increasingly connected 
global and digital society (NEST for students, 2007). 

• National Educational Technology Standards for Administrators (NETS. A): The skills and knowledge 
school administrators and leaders need to lead and sustain a culture that supports digital-age learning, 
builds a vision for technology infusion, and transforms the instmctional landscape (Standards for global 
learning in the digital age, 2011, para. 2). 

The National Educational Technology Standards for Teachers (NETS.T) has been around for more than a 
decade. However, little research is found in the literature about teacher use of technology in light of NTES.T. Sam 
(2011) examined how urban middle school teachers described their competence in the 2008 NETS.T and how they 
describe their use of technology to support teaching and learning. Participants included 45 teachers responded to the 
quantitative survey instruments and 18 teachers participated in the three focus interview groups representing three 
(private, charter, and public) middle schools. Urban middle school teachers in this study were found not aware of the 
important role technology can play in preparing students for the 2P‘ century. In addition, teachers were “not fully 
competent in the NETS.T, nor have they used them as a basis to design 2P‘ century lessons. The data show that 
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among the three classifications of schools, urban public school teachers were less aware of the NETS.T” (Sam, 

2011, p. 1 14). The researchers suggested further studies are needed to investigate high and elementary school 
teacher competence NETS.T and their use of technology to support teaching and learning. 

Using multi-stage cluster sampling of all K-12 public school teachers in New Jersey, Bergacs (2008) 
studied teacher perceptions of the alignment of their practices in using technology with NETS.T. Results found that 
144 participating teachers’ technology use was adhering to NETS.T (Bergacs, 2008). While no differences were 
found between different teaching experience groups, differences were found significant between different subject 
area groups in the adherence of teacher use of technology to NETS.T. Results indicated that there were significant 
differences between grade level groups in their technology use in light of NETS.T, with lower grades had lower 
mean scores than higher grades. The research found a significant difference between respondent groups who knew 
about, read, and understand the standards before the survey and those who did not (Bergacs, 2008). 

Statement of the Problem 

While the National Education Technology Standards for Teachers developed by International Society for 
Technology in Education (ISTE) has worked as a guide for teachers in technology implementation, lack of research 
that relates technology use to the National Education Technology Standards for Teachers is apparent in the 
literature. More precisely, no study could be found in the literature that examined teacher use of technology in light 
of the National Education Technology Standards for Teachers in Saudi Arabia. 

Significance of the Study 

Through examining teachers’ uses of technology in light of ISTE NETS.T, this study provides information 
to stakeholders in the Saudi education system, particularly since Tatweer schools are an indicator of the readiness of 
Saudi schools to implement technology assisted- progressive education that supports learner-centered approach. 
Also, with the increase in the use of emerging technologies in Saudi schools, this study provides a better 
understanding of how technology can support the learning process , as well as to assist in making modifications in 
the school environment and to develop better professional development for teachers based on a formal needs 
assessment. Such information could serve in determining how teachers use a standards-based approach to utilize 
technology in their teaching. 


Methodology 


Population and Sampling Issues 

As the leading model of the Saudi schools reform most recent initiative, Tatweer schools were found the 
best fit for the study goals, especially since the framework of the schools adopts a learner-centered approach with 
technology integration. Tatweer schools started in 2007 with 50 pilot schools nationwide and were expanded 
gradually. In 2012-2013 there were 30 Tatweer schools in Jeddah directorate. These schools were equally divided 
according to levels (elementary, intermediate, high) and genders (15 boys’ schools and 15 girls’ schools). The whole 
population (1073 teachers: 578 male and 495 female) was surveyed. 

Data Collection 

To administer the data collection process, a paper-and-pencil cross-sectional survey was used as it was distributed at 
the participating schools by the researcher. The survey contains 10 closed-ended items representing using 
educational technology based on the ISTE NETS.T and its performance indicators (NETS for teachers, 2008). The 
survey in this study uses a 4-point Likert-type rating scale, ranging from “strongly agree” to “strongly disagree”. 

Validity and Reliability 

To insure instrument’s content validity and appropriateness to study goals, an expert panel group was conducted. 

The 640 valid responses were found highly reliable as the Comobach’s alpha value for this instrument was a= .95. 
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Results 


Table 1 ISTE NETS for Teachers: Mean, Media, Mode, and Standard Deviation 



Collaborative 

knowledge 

construction 

Personalizing 

learning 

activities 

Exploring real- 
world issues 

Designing Relevant 
learning 

Practicing safe 
and legal use 
of technology 

N Valid 

640 

640 

640 

640 

640 

Missing 

0 

0 

0 

0 

0 

Mean 

3.2172 

3.0828 

2.9938 

2.8625 

2.8438 

Median 

3.0000 

3.0000 

3.0000 

3.0000 

3.0000 

Mode 

3.00 

3.00 

3.00 

3.00 

3.00 

Std. Deviation 

.77602 

.83789 

.80294 

.83159 

.82258 


Table 2 ISTE NETS for Teachers: Mean, Media, Mode, and Standard Deviation (Cont.) 




selecting 
technology 
effectively and 
productively 

Sharing best uses 
of technology 
with PEL 

Communicating 
relative info with 
students, parents, 
peers 

Locating, 

organizing, 

analyzing, 

evaluating 

information 

Interaction, 
collaboration, 
and publishing 

N 

Valid 

640 

640 

640 

640 

640 


Missing 

0 

0 

0 

0 

0 

Mean 


2.9359 

2.8813 

2.9000 

2.9609 

2.8234 

Median 


3.0000 

3.0000 

3.0000 

3.0000 

3.0000 

Mode 


3.00 

3.00 

3.00 

3.00 

3.00 

Std. Deviation 

.79353 

.83261 

.86272 

.82605 

.87304 


Data Analysis and Diseussion 

Deseriptive analysis, through reporting the mean, median, mode, standard deviation, and frequeneies, was 
utilized to summarize the use of the ISTE NETS.T in Tatweer elassrooms. Overall results showed good use of 
technology by Tatweer teachers based on ISTE NETS.T. The most frequent response found in all items was 
“Somewhat Agree”. The highest use of technology by Tatweer teachers found was “using technology in teaching to 
model collaborative knowledge construction by engaging in learning with students, colleagues, and others” (M= 
3.22, SD= .78). This use of technology included the growth of social networking tools, such as Facebook and 
Twitter, among Saudis in recent years. Utilizing these technologies in educational activities is very significant 
progress in Saudi education. The least use of technology by Tatweer teachers was in “using technology in teaching 
to help students to interact, collaborate, and publish with peers, experts, or others employing a variety of digital 
environments and media” (M= 2.82, SD= .87). Although this use was the lowest, it had a very good mean, which 
still reflected the alignment of Tatweer school teacher technology use with ISTE NETS.T, but with more variations, 
which might indicate school differences in technology use. Overall findings reflected the alignment of Tatweer 
school teacher technology use with ISTE NETS.T, which provides an indication of the positive impacts of the recent 
initiatives of the Ministry of Education to integrate educational technology into Saudi education. 

Conclusion 

As recommendations were developed from study findings in Tatweer schools in Jeddah, only, findings 
generalizability is limited. Teachers can use technology for improving the learning process through creating 
differentiated instructions to fulfill students need and meet individual learning styles. In addition, meaningful 
learning requires students to work collaboratively in knowledge construction through solving authentic problems 
that involves finding, organizing, and analyzing data, synthesizing information, and making decisions, which can be 
done in more productive and effective ways with the proper use of technology. Therefore, a standards-based 
approach to technology use in learning, based on ISTE NETS in professional development should focus on 
preparing teachers to use technology purposefully in the classrooms to develop student cognitive skills. In addition, 
stakeholders and change agents need to understand that for successful technology-assisted learning implementation 
teachers should be supported through offering ongoing and appropriate professional development programs. 
Recommendations for future studies included conducting similar study on other schools like private schools, which 
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are known to have better learning environment and technology facilities. Moreover, a qualitative study through a 
series of focus groups of selected Jeddah Tatweer school teachers is recommended to gain a deeper understanding of 
using technology uses to support a meaningful learning. 
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This study interviewed 7 teachers in a Massachusetts public middle school who already had been 
using the iPad with their students for at least 1 year. Teachers are the ones who best understand 
their students, classroom environment, and the overall schooling system. Thus, learning from 
teacher experience and suggestions is important for educational stakeholders develop an efficient 
MT integration plan. This study aimed to report teacher self-reported benefits, challenges, and 
suggestions of using the iPad in teaching and learning. The participating teachers reported three 
educational benefits and a challenge when using iPads in teaching and learning. The teachers also 
made a suggestion to overcome the challenge. 

1. Introduction 

Mobile technology (MT), such as Smartphones and tablets, combines both communication and computing 
features. People carry these portable devices anywhere to make calls and send text messages or emails to interact 
with families, colleagues, and friends. They can also use diverse mobile applications (apps), like calendars and 
schedulers in order to keep track of their daily events and work tasks. Using cloud-based file-sharing apps like 
Dropbox, people can share, save and work on files at their convenience. Mobile communication, collaboration, and 
information accessibility may give them greater flexibility in their use of time and enable people to work from 
different locations. 

With the advanced functionality of computational MT, educational leaders and technologists see the 
potential of using MT to support teaching and learning and argue that MT is one of emerging teaching tools that will 
educate the next generation (Erickson, 2012a; Erickson, 2012b). In the Horizon Report 2009, researchers reported 
that MT is a flexible teaching solution which supports learning in a variety of ways, by offering diverse new 
technological features like multimedia communication channels and a touch screen with educational applications 
(Horizon Report, 2009, p. 6). 

Previous research has shown that while MT offers many potential benefits to education, there are many 
challenges as well. In addition, recently there were reports on the negative experiences of using MT, raising 
educators’ skepticism over the educational value of MT and discouraging them from integrating it into their 
curricula. For example, the nation’s second largest school district in Los Angeles spent $1 billion to distribute the 
iPad to improve their education. However, the district recently recalled the iPad due to the series of challenges 
(Mathis, 2013). Thus, more research is needed to respond to educators’ concerns and to develop strong pedagogical 
integration strategies (Sheppard, 2011). 

This study aimed to collect, analyze, and report teacher self-perceived benefits and challenges when using 
MT, particularly iPads. This study also reports these teachers’ suggestions to overcome the challenges. The reason to 
focus on reporting the experienced teachers who already have used iPads for education is because their expertise is 
constructed based on their practice in the classroom over time. Thus, they are the ones who understand the 
classroom environment and who know how to utilize learning materials to teach students with iPads. Observing and 
understanding teacher statements can inform the field on the practical experiences of using MT from the inside 
rather than from the outside as researchers, administrators, producers or educational technologists; this folklore 
approach, insider opinions, has been proven to provide an authentic perspective that should not be ignored. These 
expert teachers’ wisdom will help educational stakeholders enhance policies, strategies, and support for teachers and 
students using iPads in education. 
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2. Literature Review 


Early adopters have examined the use of MT in the elassroom and reported its edueational potential 
(Herrington, et ah, 2009; Kukulska, 2010; Naismith, et ah, 2004; Pattern, Sanehez, & Tangney, 2006; Shuler, 2009). 
The existing researeh suggests that using mobile teehnology has been foeused on: (1) improving access to learning 
materials, (2) stimulating collaborative learning activities, (3) providing personalized learning experiences to each 
student, and (4) providing a constructivist learning environment, allowing students to develop their own learning 
resources through multimedia features. 

Mobile technology provides “anytime, anywhere"" access to educational resources to support learning 
activities. Portable MT facilitates teaching and learning independent of time or location. This educational process is 
also referred to as ‘mobile learning’ (Traxler, 2007) or ‘always-on learning’ (Cobcroft, et ah, 2006, p. 25). Using 
ubiquitous MT, students can conveniently access the Internet and find information to acquire new knowledge as they 
need it (Kinshuk & Chen, 2005). This accessibility enables students to choose when and where to study, or to study 
a subject immediately in context, increasing the productivity of learning activities (Cobcroft, et ah, 2006; Pietrzyk, 
etal., 2011). 

Second, MT facilitates collaborative learning activities. Unlike wired technology, which is constrained to a 
physical location, mobile devices facilitate the collaborative learning environment by allowing learners to share 
ideas and information beyond the classroom (Naismith, et ah, 2004). Using email and text messages, students can 
exchange their inquiry and opinions with classmates and teachers to further understand content while outside of the 
classroom. Also, when students find information that is useful for their group projects, they can record and share the 
information with their group and reflect on new knowledge together anywhere, at anytime (Naismith, et ah, 2004; 
Shuler, 2009). 

Third, MT supports individual learning experiences by providing tailored lessons to each student. The 
efficacy of individualized learning was previously discussed by educational scholars (O' Keeffe, Brady, Conlan, & 
Wade, 2006; Sampson, Karagiannidis, & Kinshuk, 2010). According to this research, individualized learning 
promotes positive learning outcomes because it customizes instruction and materials “based on an analysis of the 
learners’ objectives, current status of skills and knowledge, [and] learning style preferences”, and so helps them 
achieve meaningful learning (Sampson, Karagiannidis, & Kinshuk, 2010, p. 25). Using mobile web browsers, 
students can instantly search for information that helps their understanding. This self-remediated learning process 
help students catch up lessons and keep them engaged. By using educational game apps and monitoring the scores, 
teachers can track each learner’s progress easily, and tailor their instruction for each student’s abilities, interests, and 
needs. Also, schools can deliver on each student mobile devices the same learning content using various media, such 
as text, audio, and video. This enables students to select educational resources based on their own learning styles 
(Motiwalla, 2007; Virvou & Alepis, 2005). 

Lastly, MT encourages creation and exploration in learners and stimulates constructive learning. According 
to Jerome Bruner, who is an advocate of constructivism, learning is more effective when teachers provide learning 
activities in which students can explore and discover (Bruner, 1996) because in this way, students can actively 
construct new ideas or concepts based on their previous knowledge to scaffold new knowledge. Also, students can 
capture both “know what” and “know how” to gain a deeper understanding of both the subject and the process 
(Brown, Collins, & Duguid, 1989). This constructivist teaching approach enables learners to interact with authentic 
situations. Using portable wireless mobile technology, a teacher can promote constructive learning by creating 
active learning lessons. Using multimedia features, such as microphones and cameras, students can apply what they 
have learned in the classroom, create their own learning content, and present their understanding of the subject. 

However, along with these potential benefits, research has also reported a number of challenges in relation 
to the use of MT: accessing inappropriate information (Goad, 2012; UNESCO, 2011), classroom distraction 
(Campbell, 2006; End, Worthman, et ah, 2009; Gingerich, 2011), cost of equipping quality MT (Attewell, 2005; 
Herrington, et al, 2009; Kukulska, 2010; Thomas & Bolton, 2012; UNESCO, 2011), lack of professional 
development (Naismith, et al., 2004; Shuler, 2009; Ferry, 2009), dangerous behaviors, such as sexting (Willard, 
2011). These drawbacks are crucial factors that discourage teachers from using MT in their instruction. 

First, MT can interrupt student learning. MT enables students to access recent web resources anywhere and 
anytime; however, this advantage worries teachers because students may find incorrect information and develop 
inappropriate knowledge (Goad, 2012). Also, classroom distraction is another challenge that teachers found. 

Teachers argue that student use of mobile phones not related to classroom activities, such as texting their friends, 
causes difficulty focusing on classroom lessons and consequently risks poor learning outcomes (Campbell, 2006; 
End, Worthman, Mathews, & Wetterau, 2009). A recent research finding supports the teachers’ opinions. Gingerich 
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(2011) examined the impaet of texting on students’ eomprehension of leetures. The researeh found that students who 
reeeived text messages during the leeture performed worse on their quizzes than students who did not reeeive any 
text messages. 

Even more worrisome, MT ean also ereate interruptions that ean seriously harm learners, sueh as sexting. 
Aeeording to Pew Researeh Center (2009), 4% of student ages 12-17 reported that they had sent sexual images or 
videos of themselves to someone via text message (p.5). Fifteen pereent said that they had reeeived sexual images of 
someone they know via text message (p.5). Willard (2011), an expert in eyber-bullying, reports that eyber-bullying 
and sexting ean oeeur in the elassroom where MT is not forbidden. For these reasons, teaehers hesitate to adapt MT 
in the elassroom. 

Seeond, edueators worry about the eost to implement MT in the elassroom. The eost of mobile deviees has 
deelined while the number of mobile users has grown. Flowever, wireless network infrastrueture has not kept up 
with the advaneement of mobile teehnology (Herrington, 2009, p. 61). Speeifieally, at the institutional level, 
installing the Internet infrastrueture and expanding bandwidth to provide wireless network serviees are still 
expensive. As teehnology develops, MT has gained more eomputational eapability while types of MT are beeoming 
more diverse. Also, teaehers reported that mobile teehnologies were more useful when they were paired with other 
software and hardware deviees, sueh as video editing software, headsets, and mierophones (Herrington, 2009; 
UNESCO, 2011); however, hardware and software developments do not yet allow the diverse types of MT to 
eommunieate eheaply and effieiently. These mobile aeeessories are extra eosts that stakeholders need to eonsider. 

Third, the variety of mobile phones is another diffieulty when integrating the deviees into the elassroom. 
Eaeh phone has different eapabilities. Some phones are simple, while others are far more advaneed (Goad, 2012; 
Herrington, 2009). Thus, teaehers may be unsure how to run the same edueational aetivities for all learners beeause 
of the variation students may experienee due to deviee differenees. 

Lastly, a laek of teaeher professional development programs is another drawbaek that edueators diseuss. 
Teehnologies ean support teaehers and students, only if teaehers know “when, why and how to use these tools.” As 
with previous teehnologies, most mobile learning teehnology implementation did not inelude quality professional 
development, so “teaehers did not know how to use [MT] and as a result [MT] did not promote learning; instead it 
beeame a souree of distraetion.” To improve teaehers’ eapabilities to use mobile teehnologies, stakeholders should 
provide professional development that improves both teehnologieal and pedagogieal skills and knowledge. 

3. Method 


3.1. Data 

This is a qualitative ease study. This study eolleeted and analyzed three types of data: interviews, elassroom 
observations, and elassroom doeument analysis. 

First, I interviewed seven middle sehool teaehers experieneed in employing iPads for at least 1 year to 
investigate the benefits, ehallenges, and suggestions of using iPads. Seeond, I observed elassrooms to gain a better 
understanding of the elassroom environment and to validate teaehers’ opinions. Lastly, to understand teaehing 
proeess, I eolleeted and reviewed doeuments, sueh as lesson plans, syllabi, and other relevant resourees. Using these 
three approaehes in an investigation of using iPads triangulated data and improved the validity of the resourees. 

3.2. Participants 

There was a total of seven partieipants: two English teaehers, one Latin teaeher, two seienee teaehers, two 
soeial studies teaehers. All teaehers had already used iPads for over one year. The partieipants’ baekground 
information is deseribed in the Table 1. 


Subject 

Teaching Experience 

iPads Experience in Teaching 

English 

9 years 

Over 1 year 

English 

17 years 

Over 1 year 

Latin 

4 years 

Over 1 year 

Science 

12 years 

Over 1 year 

Science 

9 years 

Over 1 year 

Social Study 

14 years 

Over 1 year 

Social Study 

13 years 

Over 1 year 


Table 1: Teaehing Experienee of Partieipants 
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4. Findings 

All seven participating teachers reported three educational benefits of using iPads: (1) creating a 
constructive and active learning environment, (2) individualizing instructions, and (3) increasing student 
engagement. Along with these benefits, they also reported a challenge using iPads: classroom management. The 
participating teachers suggested clear communication with students regarding teacher expectation, classroom rules, 
and consequences of breaking rules as a solution to mange the inappropriate use of iPads in the classroom, called 
“iBad” behavior, and to manage classroom. Through this communication, the participating teachers aimed to 
develop their students’ responsible use of using MT in and outside of the classroom. 

4.1. Benefits 

First, all participating teachers reported that the use of iPads benefited them in creating constructive and 
active learning environment. These teachers were able to ask students to use handy apps not only to manipulate 
lesson materials, but also to create projects in the classroom conveniently. For example, students could use note- 
taking apps to manipulate and annotate their own ideas on the digital textbooks or PDF lesson handouts to create 
their own learning martials. The review of their own notes helped students to recall lesson content better. Also, by 
requiring students to develop projects using visual media with text, these participating teachers were able not only 
to develop student creativity, but also help them make connections between text and concrete examples and gain a 
better understanding of the subject matter. The following teacher report illustrates this benefit of using iPads. 

What I was having the kids do, I was having them make flash cards on their iPads in variety of 
different ways. The main method was using the Keynotes and ‘Doodle Body ’. What they will do is, 
they will draw an image using ‘Doodle Body’, and then put the main image and write the Latin 
word underneath. Students can make direct connection between words and images. It allows 
them to learn without using English. So they are not going ‘caseus ’ that means cheese, ‘malum ’ 
means apple and understanding that way [pause] ...they are directly, ‘okay I am drawing ‘caseus ’. 

This is what ‘caseus ’ looks like. ‘Malum ’ looks like that. ’ So they associate directly. 

Other teachers also provided specific examples how the iPads integration benefited participants in creating 
constructive and active learning environment. One teacher reported that by actively involving students in 
researching and encouraging them to create projects to form their understanding, students were able to construct 
their knowledge efficiently. 

We did this activity, called Genetic Disorders. So the kids have to pick a genetic disorder, and 
create a research poster on Pages based on the genetic disorder that they find out about it. They 
need to add different images, and at the bottom, they insert a video fide of a personal story. And so, 

it is really cool because they’re using Page, they have images, they have personal stories, they're 
really seeing someone’s face with genetic disorder. Kids are really, really reflective, very serious 
about it They realized, you know, just how big some of these genetic disorders are. So that ’s a 
huge one that I love. Some of them we put it on the board, and kids like to share, because it’s 
personal experience. 

Second, all participants reported that individualizing instruction is another benefit of using iPads in 
teaching. Students have different learning styles with different levels of background knowledge. By utilizing 
supplementary online resources, these participating teachers were able to support not only each student different 
learning needs, but also different student learning preferences. Furthermore, using educational game apps, teachers 
were able to provide chances for individual students to practice, reinforce, and develop skills in a certain subject 
area at their own pace. One teacher gave a specific example of accommodating different levels of student subject 
knowledge by distributing and allowing students access to different instructional materials on their own iPads. 

This [Web Quest] is great when I have to differentiate instructions to different students. So 
some students are really hungry like for the information they want to know more. So by 

accessing these websites using iPads, there are so many links within this website that they can go 
to, so the kids that work fast then they can go ahead, and get so much more information. They can 
find them on their own. If students need a little more assistance with that I also give them 
directions exactly. So sometimes I will give this other students, tell you exactly where to click, 
click on Transmission, so that students a little more help, I will give them a slightly different 
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document, other students would not be able to tell, they are slightly different, but I can give them 
few more directions, that's how I can differentiate my instructions too. Some students need more 
assistance, but maybe they don ’t want to look like they need more assistance. 

The last benefit that the participating teachers reported was the positive infiuence on student engagement. 
They believed that when students were more engaged in learning, students were more motivated to pay attention, 
work harder, and push themselves to excel. As these teachers understood it, this positive learning attitude helps 
students integrate, transfer, and retain new knowledge better. Another commented that these students’ extra effort 
led them to achieve unexpected learning outcomes, exciting teachers to use iPads in teaching. The following 
teachers’ explanation elucidated participants’ instructional belief and their feelings about student engagement when 
using iPads and its positive educational influence. 

The boy who looked up the ringworm on his iPad will probably tell his folks about it tonight. 

He's going to share that experience, because he went out of his way to find that picture and to find 
if that was virus or bacteria. Those are their mini-experiences that are different from the norm. 

They will pick up on those more. The more fun a child has with an assignment, the more likely 
they will remember it. ..In senior year they'll be like ‘Remember the day when we went to outside, 
we were jumping around leaves and had a lot of fun. ’ That's what they remember. So, you have 
to give them those experiences. 

If the kids are enjoying it, and they're having good time with it, then, it makes your job as a 
teacher a little bit easier. They're excited to use it. They're excited to work with it... That's what I 
really love about this. Sometimes you'll envision what you want the kids to do and, they'll take it 
to the next level. I gave them some guidance, but they created these things that I just wasn 't 
even expecting. They'll make something with it that you didn't even think of. Or, they'll really put 
all this extra energy or effort into it. 

The above summarize three educational benefits that all my participating teachers self-reported when using 
iPads in teaching: (1) creating constructive and active learning environment, (2) individualizing instruction, and (3) 
engaging students in learning. Along with these benefits teachers also reported one specific challenge when using 
iPads in the classroom. 

4.2. Challenge 

The participating teachers reported that classroom management is a significant issue when using MT in the 
classroom. They said that iPads have created different types of behavioral problems among students, challenging 
teachers to mange classroom than those ordinarily demanded when iPads were not in use in the classroom. 

These participants elucidated three new types of behavior problems: (1) classroom distraction, (2) student 
technical competence, and (3) no ideal punishment. First, there are many non-educational apps, such as games and 
message services that students can download and use for entertainment and social purposes. The participant teachers 
reported that students were easily distracted by these apps, which made it difficult for these teachers to maintain 
students’ focus on learning activities. In addition, because iPads are not designed for educational purposes, but for 
personal use, it was difficult to manipulate the platform and force students to use iPads only for educational 
purposes. The following example explicates participant opinions on different types of behavior problems and the 
need of different types of classroom management strategies. 

Students are not perfect. They will be off-task some days. They have good days, they have bad 
days, they have assignments they don 't like, assignments they like, subjects they like, subjects they 
don't like. Before [we had iPads], if they were off-task, they didn't have an iPad to entertain 
them, so they might poke the kid in front of them or throw paper across the room or draw 
something on the desk. You don't have as much of that happening now because there's 
something to entertain them. So, it's a different type of management. 

Second, student technological strength also challenged classroom management. Some participants 
mentioned that student technological competence helped them develop their own technological skills. Flowever, 
when it came down to classroom management issues, student technological competence made it more difficult for 
teachers to monitor student “iBad” behavior. The following teacher report explained why student technological 
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competence was a problem. 


When they are on their iPads, these kids are so savvy. They know how to sweep, you know what I 
mean. When you are walking by they are typing, you don't really know if there are closing and 
opening up another app when you walk by, you know when you come back up here. So, it's very 
hard to manage a large class to know that they are using iPads appropriately. It's not easy to 
manage. 

Lastly, participants said that there is no ideal punishment that they could apply to discipline their students’ 
“iBad” behavior. Their school suggested that these teachers could take iPads away to punish students’ misuse of 
iPads. However, these teachers explained these solutions were not effective. When they took away iPads, students 
could not complete their learning tasks because most of instructional materials and activities were on iPads. This 
dilemma is well described by the following teacher. 

There has to be more effective management because the kids get where the lines are, and they get 
the boundaries are kind of weak. One thing they understand is that they know a teacher can ’t 
really take away their iPads. Because it will be another teacher will have their homework on 
their iPads, project is due on the iPad, they need for the next class. ..They know you really can’t 
take it because they need it for other stuff. So things like that become challenging. 

In sum, the participating teachers said that iPads have created different types of behavioral problems among 
students, challenging teacher classroom management. Thus, developing different types of classroom management 
skills are needed to benefit students’ learning by using iPads. 

4.3. Suggestions 

To overcome the classroom management challenge and to guide student “iBad” behavior, these participants 
applied the communication method. They applied straight-forward communication method to explain classroom 
rules, expectation, and consequences of “iBad” behavior. The following teachers’ reports exemplified these 
participating teachers suggestion for applying straight-forward communication to manage “iBad” behavior. These 
teachers said they intended to establish their responsibility and authority through communication, to set clear 
guidelines about when, how, and what to do with iPads. 

So this year I started what 's called the responsibility rubric. So, what 's that entail? So each week 
students get 10 points for being a responsible student. So I taught them the first week what a 
responsible student is. You come in on time. You come prepared, iPad is charged. You have 
homework done, you ’re respectful, and then anytime you do something where you ’re not being 
responsible, I take a point away from 10 points you have. If you're using the iPad when you 're 
not supposed to, I let them know that you are using the iPads when you are not supposed to and 
they ’ll see me make a check on a clipboard, knowing that I am taking a point away from their 
grade. That tends to motivate them to not really want to do it. 

I think it's just those really clear limits, setting those limits from the beginning, being really clear 
about what you can and cannot do and really enforcing those becomes very important when 
you're giving a kid a device like this. Be very clear on the expectations and what you expect, and 

that you reinforce those consistently. 

You have to really make it clear when it's okay to use the iPads, and you also have to make it 
clear what the consequences are if they go against that rule. You have to warn them first and 
tell them that you should only use your iPad when I tell you to. If you use it when you're not 
supposed to, the consequence is that I will take it away, and you really have to follow through by 
doing that. One mistake by one student is all it takes, because they need to know that you really 
mean business, that it's not just an empty threat. And you will really give it to the assistant 
principle for the day. And you have to go and pick it up from her at the end of day. So they have 
to have a conversation of why it was taken away from the first place. 
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Reflecting upon these participants experiences in identifying an effective “iBad” management method, 
teacher classroom experience was to be found crucial to the process of identifying practical solutions in managing 
“iBad” behavior. Thus, to prepare teachers with adequate “iBad” discipline skills, the participating teachers 
suggested that educational stakeholders should utilize the teachers’ expertise and wisdom gained in using iPads with 
students reported in this study to contribute to developing “iBad” management guidelines. These common guidelines 
will help novice teachers who are just beginning to use MT understand the potential challenges, apply the 
appropriate solutions that match to their instructional styles, and create safe educational environments. 

5. Conclusion 

This study reported teacher self-reported benefits, a challenge, and a suggestion when integrating iPads into 
curricula. All my participating teachers reported three educational benefits: (1) creating constructive and active 
learning environment, (2) individualizing instruction, and (3) engaging students in learning. This study confirms two 
benefits that previous research reported: creating constructive learning environment and individualizing instruction. 
A new benefit that this study reports on and expands the understanding of is mobile technology’s (MT) — 
specifically, iPads — ^positive influence on student engagement in the classroom. 

Previous studies reported that classroom management is a significant issue when using MT in the 
classroom (Campbell, 2006; End, Worthman, Mathews, & Wetterau, 2009; Gingerich, 2011; see Error! Reference 
source not found.). The participating teachers also reported the same challenge. They said that iPads have created 
different types of behavioral problems among students, requiring teachers to develop different types of classroom 
management skills than those ordinarily demanded when iPads were not in use in the classroom. To overcome this 
challenge, the participant suggest that schools and teachers need to set clear mles and have clear communication 
with students about classroom expectations and the consequences of their students’ “iBad” behavior. 
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Abstract 

This investigation examined how eognitive operations in a seeond language (L2) interaet with a first 
language (LI). Forty low profieient Korean learners of English performed two writing tasks after reading an L2 text: 
writing direetly into English (eontrol eondition) as well as writing in Korean and then translating into English 
(experimental eondition). All El and L2 essays were eonverted into Pathfinder Networks. For the Pathfinder 
networks analysis, we propose “eoneept eentrality” as an innovative measurement method of knowledge strueture to 
mathematieally and visually deseribe eognitive state ehanges during El use in L2 writing. The eentrality values (i.e., 
graph eentrality & node eentrality) deseribed and analyzed the maps in order to fully deseribe and distinguish 
between the Translation and Direet groups, suggesting a fundamental way that El and L2 eognitive proeessing 
differs. 

Introduction 

Theoretically based on schema theory, conceptual networks, and cognitive structure, knowledge structure 
refers to how information elements are organized in memory (Jonassen, Beissner, & Yacci, 1993). Clariana, 
Wallance, and Godshalk (2009) proposed that knowledge structure is pre-propositional, but it is the precursor of 
meaningful expression and thus the underpinning of thought. Said differently, knowledge structure is the mental 
lexicon that consists of weighted associations between knowledge elements. Due to our dimensionally limited senses, 
knowledge elements usually enter memory as sequential (linear) strings of propositions (i.e., when reading and 
listening), but these linear strings, after consolidation with existing traces, have relational patterns. Thus in our view, 
knowledge structure is both linear and relational at the same time. 

Knowledge structure has been recognized as an important part of cognitive processing - affecting learners’ 
ability to transfer what they know to new problems and settings. In this regard, explicit knowledge structure 
externally elicited is a snapshot of the degree to which learners have organized and comprehended the content of a 
specific context. For example, an author’s knowledge structure is reflected in her text, and the reader’s knowledge 
structure becomes more like the author’s knowledge structure as he reads that text. Indeed, several studies have 
demonstrated that measures of knowledge structure can differentiate between experts and novices (e.g., 

Schvaneveldt, 1985), predict classroom learning and achievement (e.g.. Goldsmith, Johnson, & Acton, 1991), reflect 
training manipulations designed to influence learning (Kraiger, Sales, & Cannon-Bowers, 1995), and mediate 
transfer performance on novel tasks (Kraiger et ah, 1995). Thus, apposite knowledge structure seems most important 
for higher-order outcomes such as forming inferences, comprehension, and problem solving. In sum, knowledge 
structure as a demonstration of cognitive structure, the pattern of relationships among concepts in memory, is worth 
measuring. 

We utilized the knowledge structure theory to measure the lexical structure of bilingual learners, with 
Korean as an El and English as an L2. This investigation measured the flow of knowledge structure elicited as maps 
before (Premap), during (During map), and after (Post map) writing. Specially, we elicited, represented, and 
compared the knowledge structure derived from Korean English language learners’ direct (control condition) and 
translated (experimental condition) academic writing. In this investigation, we propose “centrality” as an innovative 
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measurement method of knowledge strueture (KS) to mathematieally and visually deseribe eognitive state ehanges 
during an LI use in an L2 writing. 


Background and Related Work 


LI use in L2 writing 

In the English as a Seeond Language (ESL) and Seeond Language Aequisition (SLA) fields, the value of 
the LI use in L2 writing has been a eontinuous souree of interest and debate. The bulk of researeh has been done to 
explore the use of the LI while eomposing text in L2 (Uzawa &Cumming, 1989; Kobayashi & Rinnert, 1992; 

Cohen, Brooks-Carson, & Jaeobs-Cassuto, 2000). Despite their different researeh goals, many of the studies have 
revealed that L2 profieieney may eonstrain the amount of LI use, thereby exerting varied effeets on writers’ 
eomposing proeess and the quality of writing produetion (Wang & Wen, 2002). 

Qi’s study (1998) revealed that a high profieient bilingual writer used their LI for initiating an idea, for 
developing a thought, for verifying the meaning of a word, and for avoiding working memory overloading. In 
eomparison, the lower profieieney writers used their LI far more than the higher profieieney writers in order to 
seareh out and assess appropriate wording, eompare eross-linguistie equivalents, and less frequently eonsider about 
linguistie ehoiees in the L2 (Gumming, 1990). A study eompared direet writing with translating writing reported that 
partieipants with low profieieney levels in an L2 benefited from translating proeesses from an LI into an L2, in 
terms of language use, text eontent, style, and organization (Uzawa, 1996). Another study in the eategory of 
translating writing also showed that the L2 eomposition of low profieient bilinguals written in the translation mode 
demonstrated higher levels of syntaetie eomplexity and the benefits in areas of eontent, style, and organization 
(Kobayashi et al., 1992). These writing mode studies revealed that low profieient bilinguals seem to benefit from 
translation whereas those at high levels did not. Woodall (2002) eomplieated the relation of L2 profieieney with an 
L 1 use even further, by ineluding the linguistie properties of two languages as an additional independent variable in 
his study. He found that overall, low profieieney bilinguals switehed more often from LI to their L2 than high 
profieieney writers, but this effeet was influeneed by task diffieulty and language group like non-eognate (e.g., 
Korean/English) or eognate languages (e.g., Spanish/English); switehing between a eognate language was related to 
higher quality L2 text while switehing between a non-eognate language was related to lower quality texts. 

In sum, bilinguals with different profieieney levels may use their LI in different patterns (Woodall, 2002) 
or for different eognitive proeessing while eomposing in L2. And there are indieations that both translation from the 
L 1 to the L2 and L 1 use during L2 writing tend to improve L2 text quality, but not in all situations and not for all 
writers (Kobayashi et ah, 1992; Uzawa et al., 1989). Together with the findings from the studies reviewed above, we 
propose that the use of the LI for L2 writing may bring some benefits in terms of KS like organization and 
eomplexity to the L2 writing, espeeially for lower profieieney bilinguals. Of eourse, this proposition must be 
tempered by eonsidering a number of variables sueh as the linguistie properties of two languages, the linguistie 
experienee of bilinguals, the task diffieulty (Woodall, 2002), the partieipants’ motivation to write in the languages, 
and the genre of the writing (Cohen et al., 2000). 

Although those studies provided new insights into the possible role LI plays for writing in an L2, overall, 
the value of LI use in the L2 writing has still remained a eontroversial issue. Furthermore, the reasons for LI use 
and whieh eognitive aetivities are earried out in an LI also remain somewhat unelear. As mentioned above, the LI 
ean be used to solve linguistie or lower-order problems (eognitive level), but is also used for higher-order aetivities 
sueh as planning or to prevent eognitive overload (metaeognitive level). 

To summarize, the unelear findings in LI use for L2 writing make it hard to generalize results over task or 
aeross languages, or to establish a direet link between LI use and text quality, whieh is a relevant issue for 
edueational purposes. Therefore, with a purpose to eontribute to this on-going debate, the present investigation 
attempts to provide further insight into the role LI use plays during L2 writing by applying a reeent KS analysis 
method to try to determine how the strueture of LI lexieon affeets L2 eognitive performanee. One unique way to 
unfold the influenee of an LI in L2 eognitive proeessing will be to examine L2 eognitive proeessing with low 
profieieney bilinguals. So, this investigation proposes to eapture and visually represent low profieieney bilinguals’ 

L 1 and L2 KS derived from their direet and translated aeademie writing (essays) and maps. 

As noted above, previous studies in ESL or SLA fields have not eoneeptualized KS in an LI and L2 as our 
approaeh does. This investigation, thus, presents a novel approaeh to measure bilinguals’ KS in LI and L2 in order 
to “see” the interaetion between an LI and L2, whieh ean open doors to various follow up studies. If this KS 
approaeh is supported, therefore, it eould have wide seale influenee as a eomplementary and mediating variable to 
aeeount for various language performanees and eompetenees. 
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Concept maps for eliciting knowledge structure 

Concept maps are one common approach for eliciting KS. Concept maps and other similar graphical 
representations are one way, and maybe the most explicit way, to elicit and visualize KS of content (Jonassen et al., 
1993). Concept maps in education have been the subject of intensive investigation so far. Many studies have 
validated the appropriateness of the concept mapping technique to fiilfdl different educational purposes: to support 
learning, to assess learning, and to organize and present information for teaching and learning (Arruarte, Elorriaga, 
Calvo, & Larranaga, 2012). However, as far as we can determine, no studies have investigated the validity of 
concept maps to elicit bilinguals’ KS and its influence in LI and L2 settings. Given that production tasks of words 
and sentences are notoriously difficult for bilinguals, it is important to have a cognitively easy measure that is 
comparable for bilinguals at different levels of L2 (Van Hell & Kroll, 2012). The concept mapping can be a very 
good example of such a measure that is not difficult even for poor L2 bilinguals. Therefore, this investigation 
utilized concept mapping as a tool for eliciting, representing, and comparing bilinguals’ cognitive residue in both 
languages. 

Then, which aspects or components of knowledge are captured by concept maps? Concept mapping 
consists of several distinct cognitive activities including recalling important terms (the extent of knowledge), sorting 
terms closer together or farther part (relational-associational knowledge), and linking highly related terms with a line 
and adding a linking phrase to show the meaning of the proposition in that context (propositional-declarative 
knowledge) (Clariana, 2010). According to the findings from Clariana, Koul, and Salehi (2006), concept map scores 
derived from proposition data (links drawn between terms) were more related to verbatim knowledge from the 
lesson text covering facts, terminology, and definitions; whereas concept map scores derived from relational data 
(distance between terms) were more related to comprehension, inferential, higher order knowledge. These findings 
suggest that concept map links and distances between terms represent different aspects of KS, specifically 
propositional-declarative knowledge and relational-associational knowledge. The mapping prompt used can 
differentially elicit one at the expense of the other, for example requiring hierarchical maps increases the importance 
of propositional relations and decreases the importance of the spatial relations in the map artifact (Clariana, 2010). 

In this investigation, we provided mapping directions that intentionally deemphasized propositions (links between 
terms) and that emphasized association (distance between terms) to more clearly measure and compare participants’ 
comprehension level. Thus, in this investigation, KS data consists of the distances (i.e., proximities) between the key 
terms in the maps. 

Centrality as a measure of knowledge structure 

A general graph theory concept called centrality (Freeman 1978) has been recently used by several 
researchers as a way to analyze concept maps (Ifenthaler, 2010; Clariana, Draper, & Land, 2011; Clariana, 
Engelmann, & Wu, 2013). Clariana et al. (2011) demonstrated the potential of graph centrality measures by applying 
it to an online course. They quantified four categories of network graph layout forms using graph centrality as an 
omnibus numerical measure of concept maps structure, with 0. 1 representing linear form, 0.4 a hierarchical form, 

0.6 representing a network form, and 1.0 representing a star form. Their investigation showed that the group average 
KS of the participants in a collaborative online community of practice converged toward a more hierarchical 
structure that is indicative of expertise relative to a self-paced group that did not collaborative online (Cgraph = 0.45 
vs. 0.25). Their finding supported using graph centrality for measuring convergence of KS representations in a 
computer supportive collaborative problem-solving task. 

In a follow-up investigation, Clariana et al. (2013) applied both graph centrality and node centrality (e.g., 
the number of links to a concept). They viewed node centrality as a measure of concept importance and graph 
centrality as a measure of form. Their study sought to mathematically and visually describe state changes from 
problem space to problem solution during problem solving. Participants (N=120) were randomly assigned to 
interdependent or non-interdependent conditions to work online in triads to create a concept map to solve a problem 
scenario. Node centrality of the interdependent group-created concept maps resembled the fully explicated problem 
space, while the non-interdependent group-created concept maps mainly resembled the problem solution. The node 
centrality values agreed with the result using common relations count analysis and the centrality measures allowed 
for additional comparisons including analysis by multidimensional scaling. The results demonstrated that node 
centrality provides supplementary measure of KS. In this present investigation, we compared the graph centrality (a 
single number) and node centrality (a vector) of the various maps from the translated writing group to those of the 
direct writing group to further determine the usefulness of concept map centrality as a measure of KS. 
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Purpose 


This investigation and the others cited above presumes that persistent KS exists and influences cognition in 
a way that can be expressed as a nomological network (Cronbach & Meehl, 1955, a set of constructs/concepts and 
their linkages), or at least KS influence can be measured and is worth measuring. But the existence, differentiation, 
and validation of a construct and of a model of constructs (a theory) cannot be established in one investigation, it 
requires a body of evidence. The purpose of the investigation is to add to the growing evidence of the likely validity 
of graph centrality and node centrality of maps as measures of KS by demonstrating the logical relations observed 
between these measures. 


Method 

This experimental investigation involved directed and translated writing was conducted in two face-to-face 
classes (20 & 20 respectively) with 40 Korean - English language learners at a South Korean based University in the 
spring semester of 2013. The participants all received a TOEFL (Test of English as Foreign Language) score of 
lower than 550, which is the minimum score for admission to most undergraduate schools in the USA. According to 
a self-report survey, their fluency in English is rated an average as 3 on a 7-point scale (with 7 being native-like 
fluency and 1 being elementary fluency). According to the TOEFL score and self-report, the students can be 
considered as low proficient learners of English. 

Participants were randomly assigned to either the experimental condition (translated writing) or the control 
condition (directed writing). Next, they completed mapping training lesson on how to draw maps in both Korean and 
English (a distance-related relational specific training). Then, they were asked to read a 450-word English text taken 
with permission from TOEFL reading materials and then create a map (Premap) of this content on the handout. 

Before creating a Premap, all participants were given the same list of approximately 20 key terms (but in El, Korean, 
or in L2, English depending on treatment) from the text that they could use for their Premap. They can use any terms 
and any number of terms in their maps (i.e., open-ended concept mapping). Figure 1 summarizes and compares 
activities between the two groups. 


Experimental Group (translated writing) 



Control Group (directed writing) 


RE PreME 



PostME Posttest 


RE: Reading in English, WE: Writing in English, W*-: Writing in Korean, PreME: Premap in English, PreNP-: Premap in Korean, 
DuringME: During map in Korean, PostME: Postmap in English, PostM*-: Postmap in Korean 

Figure 1. Research design 

All terms used by all participants were combined and arranged in order of frequency, and then this ordered 
list was given to a content expert. While considering this list and the instmctional text, the expert determined the 10 
essential terms {cave painting, puzzling, location, seasonal migration, hunting ceremony, tribal ceremony, 
motivation, overpainting, ancient human, intelligent) and then created an expert map with these terms, and this map 
is used as the referent map for comparing to all of the participants’ maps. 

For map analysis, Jrate (see for more detail Corsaro, 2003) software was used to calculate the raw distance 
data that contained all pair-wise distance between the 10 terms in expert’ and in participants’ maps. Each paper- 
based map was recreated in Jrate, a manual process that can introduce error, though care was taken to maintain the 
original spatial relationships between terms in the map. In this way, each map was converted to a proximity file with 
45 elements (i.e., (10^ - 10)/2= 45) that are the distances in pixels between every term. Using Knowledge Network 
and Orientation Tool software {KNOT, for more details see Clariana et al., 2006, Clariana, 2010), all of the 
participants’ and also the expert’s map proximity data (distances) were converted into network representations of 
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structural knowledge called a Pathfinder Network (PFNetj. PFNets describe the interrelatedness of concepts within a 
content domain - the knowledge structure - at that moment in time (Goldsmith et ak, 1991). Each concept is 
represented as a node in the network, and the links between the nodes are based on the pattern of distance estimates. 
Thus, PFNets are believed to show how one individual understand the interrelatedness - an organization of the 
concepts - in the certain content domain. 

All participants’ maps {represented as PFNets) and the expert referent map {represented as PFNet) were 
entered into a node degree table using the same 10 concepts. The data were analyzed in two different equations, 
using graph centrality that establishes one value ranged from 0 to 1; and then using node centrality that uses 10- 
element vector for each map that were then analyzed by correlation and multidimensional scaling (for more detail 
see Clariana et al. 2013). Since correlation values (r) are not interval-level data, the values were squared to form 
coefficient of determination values (r^) that can then be analyzed with traditional statistics. 

For essays analysis, ALA-Reader software (for more details, see Clariana et ak, 2004; Koul et ak, 2005) 
was used to transform the Korean and the English essays into raw distance data for the same 10 terms. This data was 
also converted to a proximity file and analyzed by KNOT software. Using the same approach described for map 
analysis above, all of the participants’ essay proximity data were converted into PFNets that were then entered into a 
node degree table for calculating graph centrality and node centrality. Since all maps and essays are converted to 
PFNets, then the KS elicited as maps can be compared to maps, as essays can be compared to essays, and 
remarkably, maps can be compared to essays. 


Results 


Graph centrality 

Control group’s Premap^ , writing®, and Postmap® on average were linear in form (centrality range .23 
to .28 to .34 respectively) while the experimental group’s Premap*^, writing’^’ duringmap*^, writing®, and Postmap® 
were all relational forms (.51 to .57 to .54 to .46 to .48 respectively, see Table 1). This result for graph centrality 
could indicate that the average KS of the participants in the translated writing condition converged towards a more 
relational structure like an expert (0.47) relative to the direct writing condition that had a more linear structure, 
suggesting a fundamental way that El and L2 cognitive processing differs, the latent knowledge structure of the Lx 
term set is imparted to the Lx artifacts. 


Table 1 . Graph centrality and comprehension posttest 


Comprehension Posttest 

Pre 

Graph Centrality 

During W® 

Post 

Trans. 82/100 (0.17) 

0.51 (0.12) 

0.57 (0.22) 0.54(0.30) 0.46 (0.15) 

relational graph 

0.48 (0.11) 

Direct. 64/100 (0.15) 

0.23 (0.05) 

0.28(0.11) 

linear graph 

0.34(0.18) 


(note: values shown in parentheses are standard deviation) 


As shown in Figure 2, the Direct group’s non-linear relationships between graph centrality and 
comprehension posttest for the Premap and Postmap were not significant. But the Translated group’s Premap, 

During map, and Postmap all show a significant non-linear relationship between graph centrality and posttest 
comprehension score. In other words, perhaps too little structure (which might indicate a deficient map) or too much 
structure (which might indicate non-appropriate dominance of the structure by one or a few irrelevant terms) both 
negatively influence performance on the posttest, but optimal structure (graph centrality of about . 56 for 
Premap, .52 for During map, .48 for Postmap) relates to optimal posttest scores. This result could explain in part 
why the average graph centrality values of translated group’ maps ranged from 0.46 to 0. 57, and their performance 
on comprehension posttest was much higher compared to direct group (82 vs. 64). 
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Figure 2. The relationship between graph centrality and performance on the posttest 
(comprehension score). 

Node centrality 

Proxscal multidimensional scaling (SPSS 20.0) was used to analyze the node centrality vectors including 
all of the participants’ maps and expert referent map. 
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Figure 3. The proxscal 2-dimensional representation of the map node centrality data. T indicates the 
translation group’s maps while D indicates the direct group’s maps, Pre in blue indicates their Premap 
and Post in red their Postmap, and the number indicate the individual participant ID. 


In this representation, the Expert map fell toward the left of the figure in both. The dimensional 
representations suggest the flow of map node centrality from Premap to Postmap, since more Premaps are towards 
the right and more Postmaps towards the left, we infer from the diagrams that the Translation group’s maps moved 
from the right (Premap) to the left (Postmap) of the multidimentional scaling (MDS) representation converging on 
the Expert map. The Direct group Premaps and Postmaps were fairly evenly distributed across the MDS figure with 
no clear pre-to-post shift, and if any shift is present, it is a bit more toward the right, away from the Expert map. 

Flere, convergence means that maps are more alike, and is measured as average percent overlap (Clariana et 
ah, 2013) of all maps in two conditions. To represent the percent overlap as a measure of convergence, the map data 
in the node degree table were correlated to each other within condition and then r values were squared into 
coefficients of determination (i-^). The Translation group average overlap percent was M = .66, SD = .08 and the 
average percent overlap of the Direct group was M = .32, SD = .05, effect size ofd^ 1.25 (t-test p = .01). The 
Translation group members’ maps converged more (66%) relative to the Direct group (32%). This finding shows 
that the Translation group Postmaps were much more homogenous while the Direct group Postmaps were more 
idiosyncratic. This convergence is also clearly represented in the MDS results (convergence represented as nearness 
in the MDS space). 
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Discussion 


This investigation analyzed the distanees between eoneepts in a eoneept map as a measure of KS. We 
proposed eentrality as a new measure of KS to elearly deseribe eognitive state ehanges during LI use in L2 writing. 
First, the graph eentrality values showed that the use of LI during L2 writing appears to help these low profieient 
bilingual learners who appear to have linear KS for this term set in L2 but have a relational stmeture for the same 
eoneepts in L2, and the relational stmeture had a positive influenee on their aeademie performanee in L2 (in this 
ease eomprehension multiple ehoiee posttest). This result aligns with the finding of previous researeh in Second 
Language Writing, whieh suggested that translating may bring some benefits in terms of KS like organization and 
eomplexity to L2 writing, espeeially for students at a lower profieieney level. 

Seeond, node eentrality values as MDS were interesting and signifieant in order to aeeount for and 
distinguish between the Translation group and Direet group' KS. This node degree MDS analysis demonstrated that 
the Postmaps of the Translation group from pre-to-post ‘moved’ towards the Expert map, while those of the Direet 
group were seattered and away from the Expert map (see Figure 3). We infer that having a KS more like the Expert 
leads to better eomprehension. 

To sum up, our view is that the LI stmeture is almost always rieher, more eomplex, and mueh better 
eompared to L2. As shown by this investigation, the LI ’s lexieal eomplexity strongly influenees the KS established 
in L2 and L2 text eomprehension. Of eourse, this finding does not mean that we ean eonelude that LI use during 
writing in L2 is direetly related to the good L2 writing performanee beeause L2 text quality may be positively or 
negatively affeeted by a number of variables as reported at the literature in this paper. 

Significance of the study 

This present paper deseribes a new measure of bilinguals’ KS residue in both LI and L2 established during 
mapping and writing in LI and L2, namely eentrality. In this investigation, its appropriateness and validity were 
elearly demonstrated as a useful measure of KS eontained in the bilinguals map artifaets that allow researehers to 
better deseribe and distinguish the mental representation of L1-L2 lexieal relationships. Centrality is an easy to 
automate and easy to understand measure with software or even manually. Therefore, measures and applieations of 
KS using eentrality ean be applied in online and other learning settings as an assessment tool that eould be applied to 
eoneept maps and other graphieal approaeh for elieiting, representing, and eomparing KS. 
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Adolescents’ Internet Use and Usage in a Family Context: Implications for Family Learning 
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The University of Hong Kong 

This study investigated the potential impact of the internet on the quality of family relationships using a family 
boundaries approach. A total of 825 Grade 8 students from 36 secondary schools in Hong Kong were selected 
through stratified random sampling, with levels of academic ability as the strata. Results from the structural equation 
model showed that family cohesion was significantly negatively related to internet use but significantly positively 
related to learning and information search usage. These findings provide evidence of how adolescents’ internet use 
and usage affect family relationships and advance our understanding of how technology can be used to support 
family learning. 


Introduction 

The emergence of the internet and related technologies in the home environment has fundamentally 
transformed family communication and interactions over the past two decades. Blinn-Pike (2009) examined the 
changing impact of computer technology on the family since the 1980s and concluded: “technology has proved to 
have positive and negative influences of the family” (p. 573). Little, Sillence, Sellen, and Taylor (2009) considered it 
timely “to gain a better understanding of how and why people are using and adapting communication technologies to 
suit their family lives and what effect technology can have on the interleaving of home, work and leisure” (p. 125). In 
a more recent online article, Taylor (2013) raised a thought-provoking question: Is technology creating a family 
divide? The author argued that various factors such as divergence in digital competence between children and parents 
and the independence afforded by technology have put family relationships in jeopardy. 

Family scientists and therapists have acknowledged the importance of maintaining family functioning through 
balancing certain aspects of behaviours. For instance, Olson (2000) developed the Circumplex Model of Marital and 
Family Systems to evaluate families along three major dimensions: cohesion, flexibility and communication. 
According to the model, balanced and well-functioned families should fall within the mid-range of cohesion and 
flexibility. Some studies have attempted to use the model to examine the influence of media on family well-being. 
Bryant, Bryant, Aust, and Venguopalan (2001) found some evidence that America’s prime-time television families of 
the 1990s were quite psychologically healthy as indicated by the scores reported by the families in the three 
dimensions. 

Families with adolescents are the common focus of most research since the majority of adolescents are heavy 
users of technology (Subrahmanyam & Greenfield, 2008). Increasingly, various forms of technology are being 
integrated into the daily practices of adolescents and have become an indispensable part of their lives. It is therefore 
important to consider their digital worlds “as another social context for adolescent development along the lines of 
other familiar contexts such as families, peers, and schools” (Subrahmanyam & Smahel, 2011, p. 32). The research 
question of the present study is: Does the internet help to isolate or integrate individuals in families? 

A Family Boundaries Approach 

The effects of media on individuals have been studied from the perspective of theories such as social cognitive 
theory and cultivation theory. This study adopted family systems theory to understand early adolescents’ internet use 
and usage in a family context (Mesch, 2006). The theory views the family as a basic unit of analysis with a number of 
interrelated individual components that influence each other. This conceptualisation of family enables researchers to 
examine family interactions and family processes that include “family boundaries, rules, decision making, 
independence, control, roles and communication among the components” (Goodman, 1983, p. 409). 

Family boundaries are an important feature in the theory. Rosenblatt (1994) defined family boundaries as 
“interfaces between what they bound and what surrounds what they bound. Rather than being simple barriers, they 
are gates, sieves, and windows that open and close” (p. 86). In connected families, the boundaries between individual 
members and others are relatively open, whereas in separated families, family members maintain strong, rigid and 
close boundaries among themselves. One of the indicators of family boundaries is family cohesion, which refers to 
the pattern of separateness and togetherness of family members (Olson, 2000). The arrival of the internet in the home 
presents a challenge to family boundaries, and it is thus the intent of this study to explore the potential effects of the 
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internet on the quality of family relationships using a family boundaries approach. The study extends previous work 
in this area (e.g. Mesch, 2006) by considering how parents influence adolescents’ internet use and usage through their 
parenting styles. 


The Influence of Parenting Style on Internet Use and Usage 

Despite the fact that parenting style has been conceptualised in various ways in the literature, more recent 
research has tended to use parental warmth and control to represent parenting style in the context of ICT use. Lwin, 
Stanaland, and Miyazaki (2008) showed that there was a negative association between parental control and home 
internet usage. Lee and Chae (2007) maintained that parental warmth was positively related to the use of the internet 
for educational purposes. Our previous study (Lau & Yuen, Under review) found that learning-related and leisure- 
related internet usages were positively associated with parental warmth and control respectively. 

The Influence of Internet Use and Usage on Family Relationships 

The relationship of teenagers’ internet use and usage to family cohesion remains elusive in view of some mixed 
findings reported in earlier studies. Some research shows support for the view that the internet promotes social 
interactions between members inside and outside the household. Wellman, Haase, Witte, and Hampton (2001) asserted 
that online interactions afforded by the internet heighten offline relationships as it fills the communication gaps 
between face-to-face meetings. Yoon (2006) contended that the internet allows individuals to extend family 
connections beyond their own nuclear family through regular contact between family members. Based on the Pew 
Internet and American Life Project entitled “Networked Families”, Kennedy, Smith, Wells, and Wellman (2008) 
reported that most adults found technology made their family life closer today than in the past. 

Other studies have indicated a more complex relationship between the two concepts. Mesch (2003) found that 
the presence of the internet within the home did not affect closeness between adolescents and parents in families, 
although family cohesion was significantly lowered when adolescents spent a long time on the internet (Mesch, 2006). 
Padilla- Walker, Coyne, and Fraser (2012) showed that family media use involving adolescents and their parents using 
cell phones and emails, watching TV and movies together, and playing video games together was positively associated 
with family connection. However, increased use of the internet for social networking purposes was negatively related 
to family connection. These findings suggest that the influence of media on family relationships is contingent upon 
the context and type of media used and also on whether the media are used jointly or individually. 

Method 


Sample and Procedure 

The target population comprised Secondary 2 (Grade 8) students in the 2011/2012 academic year from schools 
that implement the local curriculum in Hong Kong. The sampling frame consisted of the school identity number, 
school size for the target grade, and the students’ overall academic ability level (high, middle and low). Thirty-six 
secondary schools were sampled using stratified random sampling with levels of academic ability as the strata. This 
method ensured that students of different academic abilities were well-represented. For each sample school, two 
replacement schools were prepared so that if any sample school chose not to participate in the study, a matching school 
was available for substitution. One intact Secondary 2 class was invited to participate in the study from each sample 
school. A briefing session was organised for each class of students to explain the whole procedure of data collection. 
Students responded to a self-report online survey in about 60 minutes. This study focused on family related items in 
the survey. The final sample size was 825 after data verification. The mean age of the students was 13.16 (SD = .77), 
48% of the students were males and 52% were females. 

Measures 

Family cohesion. Students were asked to respond to four items on a 5-point Likert-type scale (1: strongly 
disagree to 5: strongly agree) adapted from the cohesion subscale of the Family Environment Scale (FES) developed 
by Moos and Moos (1986). The four items were: “Family members always do their best to help and support each 
other”, “There is an atmosphere of harmony and unity in our family”, “My family members always devoutly support 
each other” and “Family members hitherto get along well with each other”. Alpha reliability of this measure was .94. 

Internet use and usage. A single item was used to measure internet use: “In the last 2 weeks, how much time on 
average have you spent daily on using computers / the internet at home?” Six options were provided for this item (1 : 
less than 1 hour, 2: 1 to 2 hours, 3: 3 to 4 hours, 4: 5 to 6 hours, 5: 7 to 8 hours, 6: more than 8 hours). Internet usage 
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at home was assessed along two dimensions: learning-related and leisure-related. Uses and gratifieations theory was 
utilised to understand the motives for internet usage (Katz, Blumer, & Gureviteh, 1974). Aeeording to this theory, 
individuals ehoose partieular media to fulfd their different needs. Using a 5-point Likert seale (1 : never to 5: always), 
eleven items were adopted and modified from the previous studies of Slate, Manuel, and Brinson (2002) and Kalmus, 
Realo, and Siibak (2011). Exploratory faetor analysis (EFA) with prineipal eomponent method was performed on the 
items followed by varimax rotation. Results revealed two faetors with eigenvalues greater than one. The first faetor 
ealled learning and information seareh had 6 items, while the second factor called social media and entertainment had 
5 items. Further details of the EFA are given in Table 1. 


Table 1. Results of EFA on internet usage at home. 


Internet usage at home 

Factor 

loading 

Eigen value 

Cumulative 

variance 

explained 

Cronbach’s 

alpha 

Learning and information search 


3.00 

49.62% 

.78 

Discuss with classmates about matters concerning 

.71 




learning 





Do assignment / report on designated topic 

.78 




Watch news 

.67 




Read e-books 

.49 




Search for learning materials 

.82 




Search for daily life information (transportation route. 

.62 




weather forecast etc) 





Social media and entertainment 


2.47 


.71 

Download songs / movies / photos / pictures 

.69 




Play online games 

.60 




Chat with net friends (via chatroom / MSN / Skype / 

.80 




QQ) 





Browse entertainment news 

.56 




Browse social networking sites (e.g. Facebook / 

.71 




Weibo) 






Parenting style. A number of parenting style scales have been researched and used in the literature. Flowever, 
many of these scales were not designed for specific contexts and cultures. Since the purpose of this study was to 
examine the influence of parenting style on internet use and usage, the study employed an internet parenting style 
scale that was developed and validated with parents in Flong Kong. A previous study with the scale demonstrated its 
desirable psychometric properties among early Chinese adolescents (Lau & Yuen, 2013). The scale contains four 
subscales (encouragement, worry, monitoring, and permission) with 14 items rated on a 5-point Likert scale (1: 
strongly disagree to 5: strongly agree). EFA with principal component method followed by varimax rotation on the 
items indicated the four proposed factors (see Table 2 for more results of the analysis). 
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Table 2, Results of EFA on parenting style scale. 


Parenting style 

Factor 

loading 

Eigen 

value 

Cumulative 

variance 

explained 

Cronbach’s 

alpha 

Encouragement 

My parents encourage me to use computers 
frequently 

My parents think that being good at computers 
is useful for my future 

.68 

.67 

2.68 

67.74% 

.78 

My parents always talk to me about the benefits 
of computers / the internet 
My parents share their experience of using 
computers / the internet with me 

.76 

.78 




My parents always help me in using computers / 
the internet 

.67 




Worry 

My parents worry (if) using computers too often 
will cause health issues 

.79 

2.68 


.81 

My parents worry (if) using computers too often 
will decrease the time of communication with 

.77 




them 

My parents worry about my thinking ability iff 
depend too much on the internet 

.82 




My parents worry about online risks that have a 
negative impact on me (e.g. online violence, 
pornography information) 

.70 




Monitoring 

My parents always ask me what I do with 

.85 

1.70 


.80 

computers 

My parents always ask me who I chat with on 
the internet 

.86 




Permission 

My parents allow me to chat with net friends 

.81 

2.43 


.81 

My parents allow me to download songs / 

.86 




movies 





My parents allow me to play online games 

.84 





Data Analysis 

Structural equation modelling (SEM) was used to examine the relationships between the variables as shown in 
Figure 1 . Item parcels were created by averaging the items in corresponding factors (Little, Cunningham, Shahar, & 
Widamon, 2002). This practice was recommended as EFA of the items was performed to identify their factor structure 
(Yuan, Bender, & Kano, 1997). Nasser and Wisenbaker (2003) provided several reasons for using item parcels instead 
of single items to represent latent constructs. First, the distribution of item parcels is more likely to be normal compared 
with single items, and there is thus a higher chance that they meet the normality assumption of the commonly used 
maximum likelihood estimation methods. Second, the use of item parcels reduces the complexity of measurement 
models, resulting in a more precise estimation of parameters. Finally, since item parcels allow fewer numbers of 
indicators to be used in modelling, it is possible for researchers to focus on more realistic models that better describe 
increasingly complex human behaviour. 
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Figure 1. SEM of the variables 


Results 

Means and standard deviations of the variables of interest are presented in Table 3 . On average, students reported 
quite a high level of family eohesion. At the time of the survey, they had spent, on average, 3 to 4 hours daily on 
eomputers / the internet at home in the previous two weeks. They used the internet more for the purposes of soeial 
media and entertainment than for learning and information seareh. In terms of parenting style, permission was the 
dominant style, whieh was followed by monitoring, worry, and eneouragement. 

Table 3 also shows the eorrelation matrix of all the variables eoneerned in this study. Family eohesion was 
signifieantly negatively related to internet use but signifieantly positively related to the two types of internet usage 
and all the parenting styles with the exeeption of monitoring. Internet use was signifieantly positively related to soeial 
media and entertainment usage, worry style and permission style. Learning and infonnation seareh usage was 
signifieantly positively related to soeial media and entertainment usage, and all the parenting styles. On the other hand, 
soeial media and entertainment usage was only signifieantly positively related to monitoring style and pennission 
style. Most of the four parenting styles were signifieantly positively eorrelated. 


Table 3. Correlations between variables. 



Mean 

SD 

1 

2 

3 

4 

5 

6 

7 

8 

1. Family 
eohesion 

3.59 

.88 

- 








2. Internet use 

3.14 

1.42 

-.13** 

- 







3. Learning and 
information 

2.86 

.78 

.30** 

-.01 

- 






seareh 

4. Soeial media 
and 

entertainment 

3.44 

.86 

.07* 

.28** 

.28** 

- 





5. 

2.72 

.78 

.26** 

.04 

.25** 

.05 

- 




Eneouragement 
6. Worry 

3.01 

.87 

.12** 

.07* 

.16** 

.06 

.30** 

- 



7. Monitoring 

3.02 

1.10 

.02 

.05 

.20** 

.15** 

.24** 


- 


8. Permission 

3.77 

.90 


.13** 


.43** 


.04 

.05 

- 


*p < .05. **p < .01. 
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The assumptions of normality and absence of outliers were examined. Univariate and multivariate normality of 
the data were confirmed (Kline, 2005; Raykov & Marcoulides, 2008) . Based on the criterion of the Mahalanobis 
distances of all cases, 68 cases were identified as multivariate outliers and were removed from the dataset. The final 
sample size was 757. The method of maximum likelihood estimation with the option to estimate means and intercepts 
was used to handle missing data. The fit indexes (/2/df = 2.14, CFI = .99, TLI = .96, and RMSEA = .04) showed that 
the model fitted the data very well (Hu & Bentler, 1999). As shown in Table 4, permission style was significantly 
positively related to internet use; encouragement style was significantly positively related to learning and information 
search usage; parental control was significantly positively related to both types of internet usage; and family cohesion 
was significantly negatively related to internet use but significantly positively related to learning and information 
search usage. 


Table 4. Results of the SEM. 


Dependent variable 

Independent variable 

Estimate 

SE 

Standardized estimate 

Internet use 

Encouragement 

.04 

.07 

.02 


Worry 

.11 

.07 

.06 


Monitoring 

-.01 

.05 

-.01 


Permission 

.25 

.06 


R2 




.03 

Learning and 

Encouragement 

.21 

.04 


information search 

Worry 

.03 

.04 

.04 


Monitoring 

.11 

.03 



Permission 

.07 

.03 

.08* 

R2 




.10 

Social media and 

Encouragement 

-.07 

.04 

-.06 

entertainment 

Worry 

-.01 

.04 

-.01 


Monitoring 

.13 

.03 



Permission 

.45 

.03 


R2 




.23 

Family cohesion 

Internet use 

-.09 

.02 



Learning and 
information search 

.26 

.04 



Social media and 
entertainment 

.01 

.04 

.01 

R2 




.09 


*p < .05. ***p < .001. 


Discussion and Conclusion 

This study contributes to the literature on the process of family adaptation to internet technologies, which is a 
relatively under-researched area in family studies. An important finding of this study is that although internet use can 
have a negative effect on family cohesion, use of the internet for learning and information search purposes can 
positively affect family cohesion. This suggests that parents may need to employ strategies to encourage their teenage 
children to engage in such online activities. Another important finding is that there exist different pathways through 
which parenting style can influence adolescents’ internet use and usage. For example, parental encouragement of 
adolescents’ use of the internet results in more learning and information search usage and subsequently leads to higher 
family cohesion. This is an extension of previous work in this area (e.g. Mesch, 2006). 

This study provides evidence of how adolescents’ internet use and usage affect family relationships and advances 
our understanding of how technology can be used to support family learning (Hart, Bober, & Pine, 2008). In particular, 
given that the internet has become a commonplace feature in the lives of many families today, it is increasingly 
necessary to explore further how parents should participate in learning with their children using technology in the 
context of their families. 

As technology becomes less exotic and a more mundane feature of the home environment, there have been 
growing concerns over its potential influence on family interactions and relationships. For example, Lee (2009) found 
empirical support for the displacement hypothesis in that time adolescents spent on online communication displaced 
time interacting with parents but did not displace time interacting with friends. The technologisation of childhood 
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(Plowman, McPake, & Stephen, 2010) is a process that demands sustained effort from parents to mitigate its negative 
impact and promote optimal child development. Further empirical investigation in this area is definitely needed to 
inform policymakers, researchers, and parents alike. 
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Abstract 

This paper reports on the initial findings from an ongoing study exploring the impact of video self-analysis 
on preservice teachers Technological Pedagogical Content Knowledge (TPACK). Preservice teaches were instructed 
to individually analyze their videotaped microteaching simulations. Through this analysis, preservice teachers were 
able to identify their knowledge gaps, and self-prescribe specific instructional strategies to enhance their 
Technological Knowledge (TK), Pedagogical Knowledge (PK), and Content Knowledge (CK). The initial findings 
suggest video self-analysis may be a viable option in developing preservice teachers TPACK. 

Context 

Anationwide survey conducted by the Bill and Melinda Gates Foundation found that 81% of K-12 educators in 
the U.S. believe technology integration in their classroom is vital to student achievement and academic engagement 
(Bill & Melinda Gates Foundation, 2010). An overwhelming majority of K-12 educators believe teaching with 
technology is vital because numerous studies have shown that effectively integrating technology into classroom 
instruction can help teachers enhance their students learning (Beetham & Sharpe, 2013; Gulek & Demirtas, 2005; 
Wenglinsky, 1998), students’ academic achievement (Cheung, 2013; Lei, 2010; Schacter, 1999), and students 
engagement (Abrami, 2001; Wankel & Blessinger, 2013). 

As technology becomes more ubiquitous in our society, a teacher’s ability to meaningfully integrate 
educational technologies into instruction will be an increasingly vital component in becoming an exemplar teacher 
(Pierson, 2001). In order to best prepare the next generation of teachers to effectively integrate technology into their 
instruction, teacher preparation programs must do a better job preparing preservice teachers to leverage all available 
classroom technologies, in order to meet the needs of 2P‘ century learners (Ertmer & Ottenbreit-Leftwich, 2010; 

U.S. Department of Education, 2010). 

Effectively teaching with technology is a complex process, and there is a prodigious need to research and 
design new instructional strategies that will help teachers steepen their learning curve (Koehler & Mishra, 2009; 
West & Graham, 2007). This study will use the Technological Pedagogical Content Knowledge (TPACK) model as a 
theoretical framework to examine how video self-analysis impacts preservice teachers TPACK development. 

What is Technological Pedagogical Content Knowledge (TPACK)? 

Building upon Shulman’s (1986) seminal Pedagogical Content Knowledge (PCK) model, Pierson (2001) and 
Mishra and Koehler (2006) developed a new technology integration model called: Technological Pedagogical 
Content Knowledge, also known as TPACK. They believed Shulman’s (1986) PCK model was incomplete and 
hypothesized a third knowledge domain (i.e.. Technology Knowledge-TK) was needed in order for teachers to 
successfully teach in the 2P‘ century (Mishra & Koehler, 2006; Pierson, 2001). This amalgamation of TK and PCK 
became what we know today as TPACK [See Figure. 1] (Koehler & Mishra, 2009). 
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[Figure. 1] Koehler & Mishra (2009) TPACK model 

Recognizing the complexity of developing teachers TPACK, Mishra and Koehler (2006) stated that researchers 
must be flexible and open to test and develop various instmctional approaches and to be mindful that “there is no 
single technological solution that applies for every teacher, every course, or for every view of teaching” (p. 1029). 

A Potential Instructional Strategy 

A promising strategy that may help preservice teachers effectively integrate technology into their instruction is 
the use of video self-analysis. Video self-analysis has been seen as an effective tool in developing preservice 
teachers’ instructional practices, because it forces preservice teachers to see their teaching in an “objective light” 
(Sewall, 2007, p. 14), “provides actual records rather than uncertain recollections” (Kong, Shroff, & Hung, 2009, p. 
546), enables teachers to examine their instruction and develop strategies to improve future instruction (Sherin & 
van Es, 2005; Snoeyink, 2010), and helps teachers become reflective practitioners (Pellegrino & Gerber, 2012). 

Why Video Self Analysis? 

Video analysis is seen as a powerful tool for educational research, because videos are able to capture “rich 
recordings of social processes” and provide researchers with a “new source of data” (Knoblauch, Schnettler, Raab, 

& Soeffner, 2006, p. 9). Since the 1960’s, video analysis has been used as a teaching tool in teacher preparation 
programs to help preservice teachers reflect and improve their pedagogy (Moore, 1988; Fuller & Manning, 1973). In 
addition to being used to help enhance teachers pedagogy (e.g.. Chase Martin & Sadera, 2011), video analysis is also 
pervasively used in business (e.g., Hershey, Jung, Mummareddy, & Sharma, 2011), athletics (e.g., Knudson, 2013), 
and medicine (e.g., Guerlain, Turrentine, Adams, & Calland, 2004). The use of video analysis has gained popularity 
in diverse sectors because analyzing videos has been shown to help teachers, doctors, athletes, and businesses 
improve their practices by prompting key stakeholders to critically “observe, assess, and confront their own actions” 
through self-analysis (Rich & Hannafm, 2008, p.66), and develop strategies to improve practices (Sherin & van Es, 
2005; Snoeyink, 2010). 

Potential of Video Self Analysis in Developing Teachers TPACK 

In order to prepare teachers to effectively use and integrate technology into their instruction. Bell (2001) 
suggests that “teachers must first internalize the value of learning with a new strategy before they will begin to use 
the strategy” (p. 525-526). To help teachers internalize the value of integrating technology into their classroom 
instruction, researchers believe teacher preparation programs should provide opportunities for preservice teachers to 
practice teaching with technology so that they can see and experience the value of using technology in their 
classroom (Bauer & Kenton, 2005; Kent & McNergney, 1999; Willis, 2006). To help facilitate preservice teachers to 
see and experience the value of integrating technology into their instruction; the use of video self-analysis may be a 
viable solution in helping preservice teachers develop their TPACK. 

Recent studies investigating the efficacy of the use of video analysis as a teaching development tool has 
revealed that both preservice and in-service teachers improved their teaching practices when video analysis was used 
in their teacher training (Beck, King, & Marshall, 2002; Goldman, Barron, & Derry, 2014; Sherin & Van Es, 2005; 
Wang & Hartley, 2003). Studies have found that videotaping preservice teachers and having them analyze and 
reflect on their individual teaching samples, encourages meaningful reflections on their teaching practices (e.g., 
Wang and Hartley, 2003). By providing preservice teachers with opportunities to analyze their videotaped 
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microteaching simulation, preservice teachers are able to “build a stronger depth of knowledge and understanding 
about their own teaching” (Chase Martin & Sadera, 2011, p. 4300); and develop strategies to integrate technology 
into their future instruction (Chase Martin & Sadera, 2011, Pierson, 2008). 

Rationale for Study 

Over the past decade, scholars, researchers, and practitioners have shown considerable interest in designing and 
developing instructional strategies that can help teachers develop their TPACK (e.g., Hofer & Grandgenett, 2012; 
Koehler & Mishra, 2005a, 2005b). However, a review of the literature reveals that an overwhelming majority of 
these studies only employed different variations of design teams, design projects, or faculty modeling to study the 
impact of the instructional intervention in relation to teachers TPACK development (Koh & Divaharan, 2011). Due 
to the complex nature of developing teachers TPACK, there is a need to extend from our current practices and 
research alternative instructional designs (Koh & Divaharan, 2011; West & Graham, 2007). 

Although there have been prior studies looking into efficacy or impact of the use of video analysis in teacher 
preparation (e.g., Goldman et al., 2014; Kpanja, 2001; Rich & Hannafin, 2008), only two studies (i.e.. Chase Martin 
& Sadera, 2011; Pierson, 2008) have explored the use of video analysis for TPACK development. 


Author(s) 

Description of Study & Findings 

Chase Martin & Sadera (2011) 

• Qualitative case study 

• 15 graduate students 

• 12 inservice teachers; 3 higher education instructors 

• All students were videotaped one time teaching for 15 minutes 

• Students were divided up into separate small groups 

• Each group selected one group members videotaped lesson to 
collaboratively create feedback using TPACK framework as a lens 

• Students highlighted the selected videotaped lessons strengths and 
weaknesses 

• Peer analysis 

• Data: edited videos & reflections 

• Enhanced students TPACK 

Pierson (2008) 

• Qualitative case study 

• 1 1 preservice teachers 

• All students were videotaped one time teaching for five minutes 

• Students were asked to edit and analyze their teaching videos 
collaboratively with their peers 

• No guiding TPACK framework was described in the study 

• Peer analysis 

• Data: videos & reflections 

• Enhanced students TPACK 


In both studies, Pierson (2008) and Chase Martin & Sadera (2011) noted that the use of video analysis helped 
teachers develop their TPACK. Their studies discovered video analysis helped enhance preservice and inservice 
teachers TPACK by encouraging them to critically analyze and reflect on their own pedagogy, content, and 
technology (Chase Martin & Sadera, 2011; Pierson, 2008). Although these studies used video analysis, both studies 
followed prior TPACK development strategies by instructed their preservice teachers to work collaboratively in 
small work groups or design teams (e.g., Lu, Johnson, Tolley, Gilliard-Cook, & Lei, 2011). While Pierson (2008) 
and Chase Martin & Sadera (2011) found video analysis enhanced teachers TPACK, their studies did not provide 
insight into how video analysis specifically helped teachers develop their TPACK. 

Developing preservice teachers Technological Pedagogical Content Knowledge (TPACK) is a complex 
endeavor (Koehler & Mishra, 2008). Additional research is needed to understand how specific instructional 
strategies (e.g., video self-analysis) impacts preservice teachers TPACK development (Thomas, Herring, Redmond, 
& Smaldino, 2013). This study aims to describe how the use of video self-analysis during a technology integration 
course impacts preservice teachers TPACK development. 

This study will address the following question: How does video self-analysis impact preservice teachers’ 
TPACK development? 
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Methodology 


Research Setting 

This pilot study was implemented in a mandatory technology integration course at a large private university, 
located in the northeastern United States, in spring 2013. As the third and final course in a series of three technology 
integration courses, this course was designed specifically for preservice teachers who have had prior experiences 
student teaching and are entering their final academic year in the teacher preparation program. This course covered 
educational technologies such as virtual labs, mobile learning applications, learning management systems, open 
source online tools, and iMovie’s video editing software. The course took place in a technology lab that has been 
outfitted with a smartboard, a projector, and 16 brand new Mac desktop computers. 


Research Design 

In order to understand video self-analysis impact on preservice teachers TPACK development; this study 
utilized a single case study design. Single case study designs are used in research “to determine whether the 
propositions are correct or whether some alternative set of explanations might be more relevant” (Yin, 2014, p.51). 
Case studies are also effective in “catching the complexity of a single case” (Stake, 1995, p. xi) by providing insight 
into a central phenomenon (Creswell, 2002; Yin, 2014). Unlike quantitative research methodologies, researching 
through a qualitative lens enables the researcher to better understand and explain participants meaning (Morrow & 
Smith, 2000). 

Twelve preservice teachers enrolled in the technology integration course were asked to analyze two iterations 
of their videotaped micro teaching simulations of themselves teaching with technology. This technology course 
consisted of six class sessions that were distributed throughout the spring 2013 academic semester. Each session 
lasted for two hours and fifteen minutes. During sessions one, three, and four; preservice teachers are given 
autonomy to explore the four technology stations prescribed by the course instructor. Each station contained an 
educational technology that could be used to enhance teachers’ content or instruction. During these sessions, the 
preservice teachers rotate through each station where they were directed to experience, practice, and explore the new 
educational technologies. At the end of sessions one, three and four; the preservice teachers were then directed to 
reflect on their experiences exploring the prescribed educational technologies. Each session’s reflection was 
specifically designed to promote deep robust reflections (e.g., how would you specifically integrate technology “X” 
into your future classroom instruction; How would “X” technology enhance your content or students learning) from 
the preservice teachers. Before each videotaped micro teaching simulation, preservice teachers were asked to 
develop a technology enhanced 50 minute lesson plan. In designing and developing their lesson plans, the preservice 
teachers were specifically asked to think about selecting and integrating technologies into their lesson that would 
help enhance their instruction and students learning. 

During sessions two and five, the preservice teachers were instructed to teach a technology enhanced micro 
lesson. Based on the preservice teachers technology enhanced lesson plan they individually developed as part of 
their homework assignment in sessions one and four, each preservice teacher was instructed to teach a five minute 
lesson based on their 50 minute lesson plan. The preservice teacher’s five minute micro teaching simulations were 
digitally recorded by the course instructor using a flip camera. After every preservice teacher completed their micro 
teaching simulation, they were instructed to reflect on their micro teaching simulation in class. The in class 
reflection were specifically designed to have preservice teachers describe what they thought they did well, and 
identify areas they could improve their technology enhanced lesson based on the TPACK framework. 

After the preservice teachers had completed their in class reflection, they were instructed to complete a video 
self-analysis assignment for their homework. For their video self-analysis homework assignment, the preservice 
teachers were instructed to log in and access the course Blackboard site, and analyze and reflect on their individual 
videotaped micro teaching simulation which was uploaded onto Blackboard. The preservice teachers were then 
instructed to watch and analyze their videotaped teaching simulation and reflect on their experiences teaching with 
technology. While watching their videotaped teaching simulations, the preservice teachers were prompted to use 
Hofer, Grandgenett, Harris, & Swan (2011) validated TPACK observation rubric to self-assess their TPACK. After 
self-analyzing their microteaching simulation, the preservice teachers were instructed to reflect on their lesson and 
think of strategies to improve their lesson based on their lessons learned from analyzing their videotaped micro 
teaching simulation. 
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Participants 


Twelve (12) preservice teachers, who enrolled in a mandatory one credit technology integration course in 
spring of 2013, participated in this pilot study. Eleven participants were female (92%), and one participant was male 
(8%). All participants (100%) were in their final semester of their teacher preparation program, and have 
successfully completed their required field practicums. 

Data Collection 

For this study, data sources include course documents (e.g., preservice teacher reflections), videotaped micro 
teaching simulations, and observation field notes. Each participant in this study wrote seven reflections. The 
preservice teachers were then instructed to upload all assignments (e.g., reflections) electronically onto the course 
Blackboard site. All course document data was then directly downloaded from the course Blackboard site by the 
researcher. In all, 84 reflections were collected for this study. 

The participant in this study completed two micro teaching simulations. Each micro teaching simulation was 
digitally recorded using a flip camera, and each micro teaching simulation lasted for five minutes. The 24 
videotaped micro teaching simulations were then uploaded onto the course Blackboard by the course instructor. The 
researcher then accessed the videotaped micro teaching simulation data through Blackboard. In all, 120 minutes of 
preservice teachers’ micro teaching samples were collected for this study. 

Data Analysis 

The researcher examined the impact of using video self-analysis in developing preservice teachers TPACK by 
investigating whether preservice teachers Technological Knowledge (TK), Pedagogical Knowledge (PK), and/or 
Content Knowledge (CK) advanced. To strengthen the reliability of this study, data was triangulated across the 
different data sources and analyzed for emerging patterns and trends using constant comparative analysis (Boeije, 
2002; Glaser, 1964). Constant comparative analysis was used by the researcher to identify themes from the course 
documents, field notes, and videotaped micro teaching simulations. The researcher used a word processer program 
to color code and highlight the unique themes. After the themes were identified, individual memos were crafted for 
each unique theme. The memos were vital in helping the researcher elaborate the identified concepts and themes. 
Each memo included specific definitions, characteristics, and specific conditions under which the theme works. 
Propositions, illustrations, and negative cases were also developed for each theme. 

Through this analysis, the researcher explored how video self-analysis impacts preservice teacher TPACK 
through three different knowledge domains: TK, PK, and CK. Based on the TPACK framework, this study will 
describe how video self-analysis impacted preservice teachers TPACK development. The following section presents 
the results of the study. 


Results & Discussions 

This section describes how video self-analysis impacted preservice teachers’ TPACK. Using TPACK as a 
theoretical framework, the following section will discuss and describe how video self-analysis impacted preservice 
teachers TPACK through three knowledge domains: Technological Knowledge (TK), Pedagogical Knowledge (PK), 
and Content Knowledge (CK). 

Impact on TK 

All twelve (12) preservice teachers in this study indicated that the two iterations of video self-analysis helped 
them improve their Technological Knowledge (TK). This study found that the use of video self-analysis enabled 
preservice teachers to develop their TK by prompting them to observe and critically reflect on their TK during their 
self-analysis. Through this observation and reflection period, the preservice teachers confronted their own TK 
strengths and limitations; and began developing strategies to enhance their TK. The findings from this study 
corroborate with earlier studies on the value added of using video self-analysis to assess and enhance future 
instruction (e.g., Fadde, Aud, & Gilbert, 2009; Snoeyink, 2010). 

By instmcting preservice teachers to analyze and reflect on their micro teaching simulations, they were able to 
observe and identify their TK strengths and limitations. Through this reflective process, the preservice teachers in 
this study developed strategies to improve their TK and acquired a deeper understanding of how technology can be 
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leveraged to enhance classroom instruction. As one preservice teacher said: “After watching myself teach, I have 
learned that I can integrate technology to enhance instruction to make it more exciting, engaging, and relevant to 
students. I have also learned that technology can be used to simulate events that may not be possible based on 
limitations within the classroom. For example, we used the FOSS [online virtual lab] to simulate tomato growth to 
show what factors (water, sunlight, soil) would affect the growth of tomatoes. This would be especially helpful in a 
school where it may not be possible to conduct in school if the school cannot afford the supplies 

Through video self-analysis, preservice teachers in this study recognized and described specific instances 
where the use of technology enhanced their teaching and their students learning. Video self-analysis also helped 
preservice teachers highlight instances in their micro teaching simulation where they needed to enhance or further 
develop their TK. Through self-analysis, preservice teachers were able to self-prescribe specific instructional 
strategies to enhance their future lessons by integrating technology into their instruction. When preservice teachers 
were asked to describe how video self-analysis helped them enhance their TK, one preservice teacher responded: “1 
definitely improved integrating technology into my lesson fi.e., second iteration of videotaped micro teaching 
simulation] ...because the technology I used this time allowed for students to be more interactive with the 
technology ...this allowed the students to have the freedom to create their own story [e.g., using an online storyboard 
creator] Through video self-analysis, preservice teachers became more mindful of how to effectively integrate 
technology into their classroom instruction. Preservice teachers in this study also became more aware of 
purposefully integrating technology into instruction, instead of arbitrary technology use. 

Impact on PK and the Amalgamation of TK 

All twelve (12) preservice teachers in this study indicated that the two iterations of video self-analysis helped 
them improve their Pedagogical Knowledge (PK). This study found preservice teachers were able to enhance their 
Pedagogical Knowledge (PK) through observations and prescribed reflection prompts. An analysis of the data shows 
preservice teachers improve their classroom management, student assessment, lesson planning, and future 
instruction by critically observing, analyzing, and reflecting on their videotaped micro teaching simulation. One 
preservice teacher described how video self-analysis impacted her PK by stating: “I improved my students’ ability to 
use the Story Bird [i.e., online story board tool] more in my second lesson because of my improved explanation of 
how to use the program. For my second lesson, I modeled on the board how to get students to the website and 
modeled how they would start their story and how to add new pages and pictures to their story 

Through video self-analysis, preservice teachers were able to identify their individual PK strengths and 
limitations. This self-reflective process promoted preservice teachers to develop an individualized plan of action to 
enhance their PK in future instruction. Using constant comparative analysis on the preservice teachers videotaped 
microteaching simulations; this study found preservice teachers applied their lessons learned from their first video 
self-analysis to enhance their pedagogy in their second microteaching simulation. A preservice teacher describing 
how video self-analysis impacted their PK & TK stated: “I feel like I did improve with integrating technology into 
my second lesson. This is because instead of having students watch a BrainPOP [i.e., online educational website] 
video, I used technology to review content as a way to assess their knowledge through Kidspiration [i.e., digital 
concept mapping tool]. 1 felt that using Kidspiration was more beneficial and a better use of technology. " 

As preservice teachers were instructed to describe how video self-analysis impacted their PK, a unique theme 
emerged. In addition to describing how video self-analysis enhanced their pedagogy, there were numerous examples 
of preservice teachers describing how they would specifically enhance their pedagogy by integrating technology into 
their instruction. Preservice teachers in this study began to recognize the importance of amalgamating specific 
pedagogical strategies and educational technologies as a mean to enhance their classroom instruction and students 
learning. 

These findings align with prior studies, demonstrating the value added of using video analysis in developing 
preservice and inservice teachers’ pedagogical knowledge (e.g., Pellegrino & Gerber, 2012; Fadde et ah, 2009) and 
technological knowledge (Chase Martin & Sadera, 2011; Pierson, 2008). 

Impact on CK and the Amalgamation of TK & PK 

In examining the preservice teaches reflections, videotaped micro teaching samples, and observation field 
notes, the researcher discovered video self-analysis not only impacted preservice teachers Technological Knowledge 
(TK) and Pedagogical Knowledge (PK), but also their Content Knowledge (CK). The data illustrates preservice 
teachers enhanced their CK by examining their CK strengths and limitations by critically analyzing their individual 
videotaped micro teaching simulation. Through their lessons learned, the preservice teachers began to develop 
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individualized instructional strategies to enhance their CK. One preservice teacher described how video self-analysis 
impacted their CK by stating: “I did not expect to spell something wrong on the smartboard in my lesson. I wrote 
“then ” instead of “than ”... This just goes to show that even teachers make mistakes sometimes and it is ok as long 
as they learn from them... If I could go back and re-do my lesson, I would have prepared better for this lesson 

By providing preservice teachers the opportunity to analyze their microteaching simulation, they exhibited the 
ability to identify their CK gaps, and prescribe specific instruction to enhance their CK in the future. Through this 
iterative reflective process, preservice teachers demonstrated more awareness in their TK, PK, and CK; and this 
awareness led the preservice teachers to construct reflections that displayed their TPACK growth: “1 think that the 
integration of technology in my second lesson did and could have further improved. The one major change I would 
have made to the second lesson would be to complete the compound words as a class in order to further the practice 
of compound words and to go over the correct answers if students were having trouble with them. This would make 
the technology in the lesson more interactive and collaborative as a class. In addition, the smart board graffiti 
wall [i.e., online word art tool] could be extended into another activity in the lesson which would allow students to 
further their interaction with technology in the lesson. ” 

Preservice teachers in this study were able to demonstrate their TPACK growth by identifying specific 
instances where they could enhance their TK, PK, and/or CK. Through video self-analysis, preservice teachers 
developed specific instructional strategies to enhance their TPACK. In addition to demonstrating their TPACK 
growth via personal reflections, preservice teachers in this study also demonstrated their TPACK growth by applying 
their lessons learned into their second microteaching simulation. 

This study found evidence that video self-analysis may be a viable solution in helping preservice teachers 
enhance their TK, PK, and CK. By instructing preservice teachers to confront their own TK, PK, and CK via 
individualized videotaped microteaching simulations; preservice teachers in this study were able to identify their 
knowledge gaps, and prescribe specific instructional strategies to enhance their TPACK. 

Conclusion 

In this study, two iterations of micro teaching simulations and video self-analysis were implemented into an 
undergraduate technology integration course. To help develop preservice teachers TPACK, the researcher provided 
the preservice teachers with generative opportunities to analyze, reflect, and practice teaching with technology. This 
study found video self-analysis played a key role in helping preservice teachers develop their TPACK. By providing 
preservice teachers with generative opportunities to analyze their videotaped micro teaching simulations, the 
researcher found preservice teacher become more aware of their knowledge gaps (i.e., TK, PK, and CK). Through 
this increased awareness, preservice teachers in this study were able to develop specific strategies to enhance the 
three major TPACK knowledge domains (i.e., TK, PK, and CK). The findings from this study builds off and aligns 
with prior research findings of the value added of using video self-analysis and reflections in teacher development 
(e.g., Amobi, 2005; Sherin & van Es, 2005; Snoeyink, 2010). 

Participants in this study were able to identify their knowledge gaps by leveraging their lessons learned from 
their prior technology, pedagogy, and content courses. Since the participants in this study were in their final semester 
of the teacher preparation program, and had extensive field practicum experiences; they were able to utilize their 
knowledge base to construct a technology enhanced lesson. Based on the finding from this study, it appears video 
self-analysis may be a viable option in developing preservice teachers TPACK. 

However, further research is needed to better understand how video self-analysis specifically impacts 
preservice teachers TPACK development. Acquiring a deeper understanding of how video analysis impacts 
preservice teachers TPACK development, can help inform teacher educators and teacher preparation programs with 
a new instructional model that may help ascend preservice teachers TPACK. 
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Abstract 

The flipped classroom model is an instructional model in which students learn basic subject matter prior to meeting 
in the classroom. Students come to the classroom with pre -class work finished, and prepared for active learning 
experiences. Previous research has shown that the flipped classroom model can 1) motivate students toward active 
learning, 2) improve their higher-order thinking skills and, 3) increase their collaborative learning skills. However, 
investigation into instructors’ perspectives on using the flipped classroom model in instruction is lacking. This paper 
is a qualitative case study on instructors’ experiences and perspectives on using a flipped classroom model in 
instruction. Structured interviews were conducted with eight faculty members who had completed a university 
training program on using active learning techniques and who either used or planned to use the flipped classroom 
model in their courses. The interview data showed different approaches by the instructors with regards to the 
implementation of the flipped classroom model in instruction, different perceived challenges, and showed the 
perceived need for outside support. 


Introduction 

The flipped classroom, which is also referred as an inverted classroom, is an instructional model in which 
the learning content is not presented during the in-class time (Strayer, 2007; Baker, 2000), but rather is learned by 
students prior to the classroom meeting through various forms of media (Bland, 2006; Foertsch, Mose, Strikwerda & 
Litzkow, 2002). This frees up in-class time for active learning, like practice exercises, problem solving, hands-on 
work, and collaborative projects (Baker, 2000; Lage, Platt, & Treglia, 2000; Zappe, Messner, Litzinger, & Lee, 

2009; Demetry, 2010). In contrast to a traditional, lecture-based, instructor-centered instructional model, the flipped 
classroom model is composed of two phases of instruction that are “flipped”, “inverted”, or “reversed” from the 
typical sequence used in classroom instruction (Bergmann & Sams, 2012). 

The first phase in the flipped classroom model is the pre-class learning phase. In this phase, basic subject 
contents and concepts are acquired by students before class. Students learn by viewing instructor provided learning 
materials in various media formats, such as online videos, podcasts, or text-format materials (Baker, 2000; Strayer, 
2007; Strayer, 2012; Bergmann & Sams, 2012). 

The second phase is the in-class learning phase. Because the introduction to basic knowledge content 
occurs in the first phase, in this phase student-centered active learning activities occur. Examples of these activities 
are interactive lectures, problem solving, laboratory experiments, collaborative designs and project creation 
(Gerstein, 2011; Strayer, 2012). A flipped classroom course can be taught in various physical facilities, not only a 
traditional lecture hall, but also in the technology-enhanced classrooms, in the studios, laboratories, computer labs, 
meeting rooms, outdoor settings, or in online learning spaces, such as Blackboard. Recent research indicates that 
students can be kept highly engaged when using the flipped classroom model (Gehringer & Peddycord III, 2013; 
Dove, 2013). Students’ motivation is increased by real-world, in-class, active learning activities (Zappe et al., 2009). 
A key advantage of the flipped classroom is that students can control their learning in terms of the learning pace and 
mastery of content (Alvarez, 201 1). Research has shown that students take the responsibility for their learning, and 
come to the class better prepared than in traditional lecture approaches (Alvarez, 2011; Fulton, 2012). Research 
using post-class surveys also demonstrates that students perceived that the flipped classroom offers a unique, yet 
challenging opportunity to maximize learning effectiveness (Wagner, Laforge & Cripps, 2013; Demetry, 2007), and 
helps to improve problem-solving and application skills (Zappe et al., 2009). Gertein (2011), Stuntz (2013), and 
Strayer (2011) reported that the flipped classroom model can provide an interactive atmosphere, which can improve 
students’ in-depth communication and collaboration (Gerstein, 2011; Stuntz, 2013; Strayer, 2012). 

Other research shows that the flipped classroom has a positive impact on students’ learning (Gerstein, 

2011; Strayer, 2012; Dove, 2013), demonstrated by increased student engagement in learning, increased higher- 
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order thinking skills, increased problem-solving skills, and improved collaborative skills (Demetry, 2010; Bergmann 
& Sams, 2012; Wagner, Laforge & Cripps, 2013; Stuntz, 2013; Strayer, 2012). However, from literature review, the 
authors found that current studies examined the influence of the flipped classroom on students’ attitudes, 
perceptions, and experiences. These research contexts have been limited to one specific course in each case. No 
studies examined instructors’ perceptions or experiences of using the flipped classroom model for instruction. 

The purpose of this study was to discover how instructors define the characteristics of “the flipped 
classroom” instructional model, why they chose to use this model in their courses, and to discover their thoughts on 
the advantages and challenges associated with using the flipped classroom model. A qualitative case study approach 
was employed to investigate these instructors’ viewpoints. 

This study examined the following questions: (a) how do instructors define “the flipped classroom”; (b) 
why did instructors choose the flipped classroom model; (c) what key factors are associated with the effectiveness of 
the flipped classroom model; and, (d) what support did instructors view as important for successful implementation 
of the flipped classroom model? 


Methodology 

A qualitative case study method was used in this study, in order to gain intensive, rich, and in-depth 
analysis. This approach was selected as the research methodology because it can provide researchers with a deep 
understanding of the multiple aspects of a phenomenon in a natural setting (Yin, 2003). Further, it allows a holistic, 
more comprehensive understanding of meaningful context (Punch, 2005). 

Participants 

Eight participants were selected from 18 faculty members who attended in a Summer Teaching Institute 
sponsored by the Teaching and Learning Center at The University of Tennessee at Knoxville, and asked to volunteer 
in this study (Table 1). In the Summer Teaching Institute, a series of workshops on integrating technologies in 
instruction and student-centered active learning instructional strategies were provided for faculty members. Four 
faculty members who had implemented the flipped classroom model (F) in their courses, and four faculty members 
who were preparing to use the flipped classroom model (PF) in the coming semester were selected as the 
participants of this study. Additionally, these eight participants were from seven different academic disciplines 
(Table 1). This diversity enabled the researchers to have a more comprehensive investigation into the stories and 
viewpoints of university instructors at UTK. 


Table 1. Summary of the participants 


Flip model status 

Course 

Code 

Flipped (F) 

Foreign language 

FI 


Foreign language 

F2 


Business 

F3 


Animal Science 

F4 

Pre-Flip (PF) 

Mathematics 

PFl 


Architecture 

PF2 


Statistics 

PF3 


Forestry 

PF4 


Interview 

An individual structured interview was conducted with each participant. Table 2 contains a list of the 
interview questions. 


143 




Table 2, Interviewing Questions 

1. What lessons did you take from your participation in the Summer Teaching Institute? 

2. Elaborate how you implemented the flipped classroom model, or your plan of using it. 

3. What difficulties have you encountered, or foreseen in implementing the flipped classroom model? 

4. Tell me about the changes in time management you’ve made with relation to the flipped classroom model. 

5. What is your present definition of the flipped classroom model? 

6. In your mind, what concepts or characteristics of the flipped classroom model are essential? 

7. As an instructor, what advantages or disadvantages could you see of using the flipped classroom model? 

8. Wliat, if any, resources or outside support, other than the Summer Teaching Institute, have or may influence 
your thinking about the flipped classroom model? 


Specifically, participants were asked to define “the flipped classroom” according to their own experiences 
and understanding. They were also specifically asked their perceptions of the advantages and challenges associated 
with an actual or planned implementation. Additionally, they were asked to comment on what support they thought 
was useful in using the flipped classroom model in their courses. 

Data Analysis 

Interview data was audio recorded and transcribed into computer files, initially hand-coded, and then 
computer coded using NVivo software with identification of recurring phrases and regularities in the data (Miles & 
Huberman, 1994). All initial codes followed closely the concepts used by the interviewees to enhance the validity of 
this study (Punch, 2005). Each transcribed file was analyzed using the set-up codes, and new codes were added as 
new concepts were generated. The transcripts were analyzed again after the revisions of all the coding categories, a 
comparison was conducted in each category between different participants. Major themes were developed based on 
the comparison of codes regarding to different participants in the same category. 

Findings 

Perceived “flipped classroom” Definition 

All eight participants generally understood “the flipped classroom” to be a teaching and learning model in 
which the subject knowledge concepts, which are the lecture material, are completed outside of class for students’ 
self-directed learning. After this “prior knowledge” is acquired students come to class to apply what they have 
gained from pre-class learning and do so in a more interactive and collaborative way than a typical lecture format. 
The following statements represent several participants’ definitions of the flipped classroom: 

“What I think is the flip model is one in which the students not only study the verb conjugations at home 
but also study the grammar at home, and they come to class to do their homework. (FI)” 

“The flipped classroom is to use the class time doing exercises instead of myself standing in front of them 
repeating what they could have read in the book. (F2)” 

“The flipped model to me means the student does a lot of the groundwork outside of class and they bring 
their basic knowledge of the material to class, and then you actually have active learning, where they are engaged in 
using that material to solve problems.” (PF3) 

“The flipped classroom is a model in which pushing out information [outside class] and putting more 
ownership on students, and students come in at a level of preparation. It's a bit more than just reading the material, 
because in many cases reading is not the level of comprehension that you want. You want them to already start 
thinking about the integration of facts and concepts, and then come into class. (F3)” 

The participants had different focuses regarding the flipped classroom. F2 focused on the instructor’s role 
change in the flipped classroom and emphasized that the instructor is no longer the transmitter of knowledge in 
class. PF3 focused on students’ engagement in problem solving in class with the knowledge base gained during pre- 
class learning. F3 focused on students’ responsibility for their learning, and emphasized students’ preparation before 
class. All of the participants placed emphasis on the students’ acquisition of basic knowledge learning before class 
and the use of in-class active learning activities. These activities emphasize students’ enjoyment and knowledge 
application skills in the classroom setting. 
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The instructors described how they would implement the flip model in their courses. A common aim was 
to transfer students’ passive learning to active learning, and to create more time and opportunities to apply the 
knowledge gained during the in-class sessions. 

Foreign language instructors, FI and F2, had students learn vocabulary before class. This is common for 
foreign language instruction and “similar” to a typical approach in the flipped classroom. They “flipped” their 
courses in order to give students more in-class time for “high level activities”, such as “oral translation” and “writing 
exercises”. Lower levels of learning such as “recognition” vocabulary and grammar would have occurred prior to 
coming to class. PF2, an architecture instructor, indicated that his/her aim was “to change students’ passive learning 
style and to encourage students to engage in complex projects having as many 500 variables”. P3, who taught 
different sections of a supply-chain business course along with several other instructors, expressed the aim of the 
flipped classroom as: 

“We want students to be able to apply concepts and to make a very different level of learning. We want 
students to read, comprehend at a certain level and then come into class, and we'll start the discussion not about what 
the definition is, but really start talking about it in application, strengths, weaknesses and so forth; incorporate 
technology in instruction (P3).” 

P4, an instructor in animal science, had previously implemented a “partial flipped course”, in which only a 
proportion of the course was implemented as a flipped classroom approach. The flipped part of the course was in an 
online format. The students’ pre-class learning assignment was to upload their presentations on a discussion forum. 
The in-class time was for face-to-face intense discussion of the pre-class uploaded online presentations. P4 
perceived it as a way to “allow for much deeper delving into the learning content than using the class time to go 
through the presentations”. The students “had no longer listened to others’ presentations as audience”, but “engaged 
into more intensive and detailed discussions about what each presentation said.” 

Moreover, all the instructors viewed improving students’ self-directed learning skills as an additional aim 
of using the flipped classroom model in their courses 

These instructors’ perceived advantages of using the flipped classroom model in instruction could be 
summarized as: (a), freeing in-class time for students’ active learning, (b) enabling students into an more interactive 
and more in-depth way of learning, (c) improving students’ skills in application and problem solving, and, (d) 
improving students’ skills in collaboration and self-directed learning. 

Perceived Approaches for Making an Effective Flipped Conrse 

Interview data revealed four major components instructors should do to make an effective flipped course. 
These were: (a) ensure students have prepared for in-class session prior to class; (b) have organized instructional 
design with good structural components; (c) design learning materials and activities based on students’ feedback, 
and, (d) provide instant support and scaffolding of learning in class. 

Ensure Students Have Prepared Prior to Class 

All eight instructors highlighted the importance of students’ pre-class learning. They stated that students 
needed to complete assignments in pre-class learning to ensure that they could get involved in the in-class active 
learning activities. 

“It is a biggest issue to have students do the work outside class, so that they are prepared when they come 
into the classroom. . . If the students can do the work outside class within the lecture content, it means we 
[instructors] can devote a whole class to be in the field, developing vegetation sampling plans and wildlife 
monitoring plans. . .If the students haven’t done those, they are going to find it is very difficult to follow classroom 
content. (PF4)” 

“. . .want them [students] to already start thinking about the integration of facts and concepts. . .when they 
come into class if they've [prepared], if we've done the Flip correctly, that's kind of where we start. (F3)” 

“That students really have done the pre-class work is essential to the flipped classroom model. Students 
need to learn the basic knowledge, not only reading the materials, but also answering the questions. . . I don’t feel it 
is going to be hard to carry out the in-class active learning, if the class is pre-organized, pre-class materials are 
already delivered, and make sure the students have learned before class, then come to class prepared. (PF3)” 

Instructors indicated that in order for the flipped classroom model to be effectively implemented, students 
must do the prerequisite learning phase before coming to class. Instructors identified this as the “biggest issue”, 
“important”, and “essential”. Instructors believed that the pre-class learning equipped students with basic concepts 
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required for in-class active learning. The in-class sessions provided students the opportunities to apply, integrate, 
and interact the knowledge gained from pre-class learning. 

In order to ensure students learned before class, all the participants indicated that requiring students to 
finish some pre-class assignments was necessary. In addition to quizzes, participants shared other approaches they 
used for pre-class learning assignments, such as asking students to take notes when viewing pre-class learning 
videos (PFl), posting writing samples before class (F2), and posting presentations online for instructors and teaching 
assistants to review (F4). F3 used the “carrot and stick” approach to increase student completion of pre -class 
assignments. The scores of the pre-class quizzes were calculated into students’ final grade. Students would lose the 
grade if they did not finish the pre-class quizzes. The grade jeopardy provided the “stick”. These quizzes were not 
difficult, so if the students had gained the knowledge through pre-class learning, it would be easy to get this 10% of 
the final grade. In this sense, it was a “reward”, or “carrot”, that brought students to a good level of preparation for 
the in-class phase. 

F3’s statement corresponded with the summary of PFl, who was a pre-Flip instructor, planned to use the 
flipped classroom model in the coming semester. PFl commented that the pre-class assignment was necessary. PFl 
planned to take the grades of students’ pre-class assignments as a part in the final grade of the course, with the aim 
of promoting students to be prepared prior to class. However, PFl suggested that the pre -class assignment should 
not be too difficult, or take too much time for students to complete. 

Be Well-organized in Instrnction 

All the instructors commented that in comparison to the traditional instructor-centered, lecture -based 
model, the flipped classroom model required them to be better organized in instructional design and implementation. 
PF2 commented: 

“You [instructor] need to be very well organized, have very clear objectives as to what they [students] are 
to do prior to class. (PF2)” 

PF2’s comments focused on clear organization when designing and assigning pre-class assignments. PFl’s 
comments further illustrated the importance of a well-organized structure in the flipped classroom instruction: 

“Structure, schedule of the course, activities, group arrangement, must be well-organized. . . You have to 
do a schedule. There has to be some organization in class during the group activities so that it's not just chaos. I've 
got to organize how I'm going to do that and just how we're going to get started each day. (PFl)” 

PFl indicated that instructors should be well-organized in all aspects of the flipped classroom instruction, 
such as the syllabus, schedule for activities, activities design, and grouping arrangement for the in-class 
collaborative work, in order to avoid chaos in class. This was in agreement with F4’s comments: 

“You need to be organized, you need to plan ahead. (F4)” 

Moreover, F4 comments on good organization on students’ evaluation: 

“There is a deadline for submitting the presentations, then five days for reviewing and evaluating each 
other’s group presentations. . .The evaluation of this course was rubric guided. (F4)” 

When designing and implementing the flipped classroom instruction, the instructor should have clear and 
detailed organization of what the students should do for the complete semester. Students should be guided by the 
instructor to know when they would have access to pre-class learning materials before a specific day, the deadline 
and requirement for submitting the pre -class assignment, the in-class activities requirement, grouping assignment, 
and how their learning would be evaluated. 

In addition, F3 suggested that instructors should take the organization of students’ responsibilities 
assignment in group work into consideration: 

“Success is marked by students taking responsibility. . .making sure that the students understand what their 
responsibility is. . .makes it [instruction] successful. Collaboration and shared learning are essential components, 
characteristics in Flip” (F3). 

Based on the interviews, none of the participants expressed concerns on how to be well-organized in 
instruction, but indicated that it was not difficult, just to be careful in scheduling the instrnction. PF2 seemed to be 
very confident in organizing the class by commenting that he/she felt “really good in standing up and orchestrating a 
jazz band”. PF2 indicated it was much better to have a general plan of the whole course, which would enable the 
instructor to respond to whatever came up in class. 
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Think as Students 


Six of the participants indicated that when they considered the instructional design of the courses to be 
implemented in the flipped classroom model, getting to know more about students was important. Finding out 
students’ characteristics and needs, viewing instraction from students’ viewpoints, and getting feedback from 
students, would play a critical role in making instruction more effective. 

Awareness of the students’ needs motivated F3 and colleagues to think of using the flipped classroom in 
the supply-chain business course they taught. Comprehensive and deep probing into the learning styles and needs of 
students, described as “digital natives”, provided them guidelines for their instructional design. F3 explained that the 
instructors learned to “think as students”. They found that the students, who were better in using technology, got 
bored on the exams, and they were good in searching and communicating the exam answers, so approaches which 
could improve and test students’ abilities in application and problem solving were necessary. The activities should 
not only “make students a different level of learning” (F3), enabling higher-order thinking, but also reduce students’ 
workload, because most students “had to work more than 10 hours a week” (F3). As a result, instructors should 
build short time exercises to help students to learn and apply knowledge. F3 also suggested that instructors should 
“really start digging a little bit deeper” on how students prepare for class, how much effort they invest, and whether 
they know how to prepare for class. 

FI and F2 both suggested that when preparing the pre-class learning materials, instructors should look at 
the knowledge from students’ perspectives. 

“Instructors should look at them [pre-class materials] from a slightly different perspective, because students 
will learn when the instructor is not with them” (FI). 

“Instructors always think some knowledge is simple, but not to the students. The instructor does not 
recognize it. Especially when instructors create videos, they don’t have the students to judge if they can get all the 
knowledge understood via viewing them” (F2). 

F 1 and F2 suggested that when preparing pre-class materials for the flipped classroom, instructors should 
think of students’ current knowledge level, and design the pre-class learning materials that students can understand. 

Additionally, PFl suggested that the pre-class videos should be kept short in order to keep students’ 
attention on pre-class learning: 

“ . . . doing little quick things just to grab students ’ attention, or just a little brief lesson without 
overwhelming them with information” (PFl). 

Most participants suggested that instractors should actively collect students’ feedback on instruction. For 
example, F2 and F4 gave students’ surveys during the semester to get their comments on pre-class learning materials, 
and made changes to improve instruction, PF3 “monitored students’ feedback on discussion board” in order to make 
revisions on instruction. 

Provide Instant Snpport in Class 

All participants claimed that a big advantage of the flipped classroom model was that students could have 
instant support during in-class exercises. Illustrations of instant support in class are described below. 

“Students can have instant help because the instructor, GTAs, and other students will be in there. So to me 
that seems like a huge advantage” (PFl). 

“Students [will] practice more with support right there; they can actually work through problems and they 
are not trying to struggle through them on their own in their dorm at night with no help available” (PF3). 

An instructor in the design stage of the flipped classroom, PF3, believed the application of skills would be 
improved over that of the students in traditional lecture -based model. This is because students would practice more 
and receive more immediate guidance and scaffolding in a just-in-time situation. PFl mentioned this as “a huge 
advantage” highlighting the importance of instant support in class from not only the instructor, but also GTAs’ and 
other students. 

“Some exercises are difficult. I plan to challenge students, they have to think, they have to analyze. I work 
when them only if they have problems or instruction is not enough” (F2). 

F2’s comment demonstrated the rule of providing students instant support in class. PF2 felt that the 
instructor’s support should motivate and guide students to think and explore by themselves. PF3 argued that it was a 
challenge for the instructor to “balance providing students freedom to be responsible for their learning and giving 
them support”. 

PFl focused more on instructor’s interaction with students: 
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“I really like to have interaction with students. I think FLIP will allow me to have interactions a lot more 
both with the class as a whole but then with groups of students and one on one. .. in-classroom activitie is particularly 
appealing to me, rather than [standing] in front of the classroom and just lecturing” (PFl). 

PFl ’s instant support was similar to an in-class interaction with students. Instructors could flexibly interact 
with a whole class of students, interact among different groups when students were having collaborative work, or 
interact with a specific student. 

Perceived Challenges 

In the interviews, the participants were also asked to share challenges they had or foresaw with the flipped 
classroom model in their courses. The following are representative responses from the participants. 

Motivate and Make Snre Stndents Have Learned Before Class 

As mentioned previously, a critical factor for making an effective flipped course is that the students have 
learned the knowledge in the pre-class learning. All of the instructors were concerned with motivation of students 
toward preparation and learning before coming to class. 

“The main obstacle is the fact that sometimes students do not read the material before class” (FI). 

“My biggest concern is students [might] come to class without doing the video lecture at home, not be 
prepared enough to do whatever I have planned on that day, and get stuck in class” (PFl). 

"I have tremendous trouble getting them to do assigned readings. . .Some of the work will be assessed, but 
some not, so how to generate incentive for students to do the outside classwork becomes a big challenge” (F2). 

“I felt disappointed because it (the flipped classroom) put a lot of responsibility on students, but they show 
up not prepared. . .students have never been or really expect to be held accountable to that level of preparation 
. . .students will get lost, frustrated if they come into class without preparation” (F3). 

Expressions such as “main obstacle”, “biggest concern”, “a big challenge”, and “tremendous trouble” 
illustrated instructor’s concerns they had about a lack of student engagement in the in-class learning activities. This 
was attributed to a concern about the possible lack of pre-class preparation by students. 

F3 described that the instructors were very disappointed upon finding students “not accountable to the level 
of preparation” prior to class. This reduced the instructional effect of the in-class active learning activities. Students 
were not able to keep up and participate in the activities which required the knowledge that was supposed to be 
acquired before class. 

During the interviews, several instractors shared their strategies to motivate students to prepare before 
class. These strategies included, checks of students’ basic knowledge learning though the use of clickers at the 
beginning of class, requirements of students to post their foreign language writing journals online, and calculating 
students’ pre-class quiz scores and note taking performances in final grades. F3 also shared the experience of letting 
students know how to prepare for in-class sessions and the amount of time that should be invested. 

Not All Students Like Active Learning 

Two instructors indicated that it was a challenge for them to implement the flipped classroom model 
because not all the students liked active learning format. Students, in general are used to a passive learning format. 

“They are more used to the traditional lecture in which instructors stand in the front of the classroom and 
explain everything. They don’t ask questions when they have difficulties” (F2). 

How to turn passive learners into active learners remains a challenge. This is especially true when 
instructors did not know whether the students who kept silent even if they had difficulties in learning, had acquired 
the required knowledge or not. 

Instructors described ways they tried to motivate these passive learners to engage in active learning. F2 
indicated that instructors should try to convince students from the beginning of the semester that active learning is 
good for them. In practice, F2 focused more on how to acquire “silenf ’ students’ feedback. F2 suggested that it 
would make these students feel more confortable if the instructor could give students feedback about the questions 
they asked in the journals they submitted, so instead of asking directly in class, students were “forced” to submit 
journals every week. PF4 in anticipation of flipping the classroom believed that there might have been a few 
students not used to it, because “the flipped classroom is new to students”. PF4 suggested that instructors should 
“gauge how students are receiving it [the flipped classroom model]” first, then “doing [implement] it[the flipped 
classroom model] in gradual stages”, giving students more and more responsibilities in the in-class learning 
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activities. The staging approach would give the instructor more time to evaluate how well students were buying into 
the idea the flipped classroom model. It would also help the instructor gain from the experience in how to improve 
future instruction. 

Improve Students’ Collaboration 

Although the participants commented that the in-class active learning activities could bring students into a 
more in-depth learning, motivate students’ learning interest, and provide convenience for students to refer to help in 
class, several instructors were concerned that students did not collaborate well in the in-class active learning 
activities. 

“My challenge now is to create activities that force them [students] to work together, but some students 
would choose to do [the course work] by themselves” (PFl). 

A concern of PFl was that the students would not collaborate in class because they were used working 
alone. PFl planned to use the “jigsaw technique" to promote collaborative activities in the classroom. The “jigsaw 
technique” requires students to do something on their own and then to work together. The aim is “getting everybody 
working together” and “learning from each other”. 

In contrast to PFl, FI required more of students’ collaboration. FI felt the jigsaw method of putting 
students’ individual works together was a low level of collaboration. This was described as “working on their own 
and simply putting individual work together on their own, but never coming back for reflection, revision, or 
reviewing others’ contributions.” FI expected students to learn not only how to do work, but also learn how to 
integrate, revise, and reflect on the work and learning done. 

Huge Time and Effort Investment 

All instructors indicated that time and effort investment on preparing using the flipped classroom in 
instruction was an additional challenge in comparison to traditional lecture, especially when instructors implement 
this model for the first time. 

“Time is a major problem because I had to create the videos and at the end of the semester, but I didn't have 
time to finish” (F2). 

“Making videos involves lots of time and effort to prepare, such as writing script, and practice. . . A lot 
more pre-thinking has to be used to put what you already do into an online video, which leaves out all the questions, 
and to provide the student what you might do in a lecture plus Q and A or in a hands-on environment. . .You must 
have lots of thinking about the outcomes and how to structure things in order to create those outcomes, but then be 
able to verify in some way whether they were achieved or not. . .Lots of more things to improve the course, more on 
editing the videos, and for future use” (PF2). 

“Time is a big limitation for a new faculty member teaching four classes to prepare for FLIP, particularly 
the videos” (PF4). 

The above statements demonstrate some instructors thought video creation was costly in both time and 
energy. This was true with both the process of technical creation and revision and also with the designing of the 
videos. Thinking and reflecting about what to put into the videos, how to combine the videos with the in-class 
activities, what the instructor expects students to achieve and how to help them achieve with the videos, all required 
large quantities of time and energy. Additionally, revising the videos to improve the instruction for future use would 
require yet more time. 

“One of the things I would like to do but [would] not have time completely is to create mechanics to make 
sure that they do what they need to do” (PF2). 

“For a fairly new faculty member, I think at least initially for me it’s going to be a pretty steep learning 
curve” (PF4). 

“From a time standpoint it [the flipped classroom] can consume you. I can't imagine if my department 
hadn't given me two years to set this course in place and get those things. It is a learning curve about really knowing 
how much is doable, hoping this semester my time investment is going to be less” (F3). 

The phrase a “steep learning curve” was used by PF4 to express concern for the anticipated time and 
energy investment in learning and exploring to use the flipped classroom model in instruction. Similarly, F3 was 
astonished with the amount of time needed to initiate the flipped classroom model. However, F3 illustrated that it 
was a learning process for instructors and hoped that future time investment might be decreased with future classes 
and more time in-class would be freed up due to flipping. 
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“[Instructor’s time management] is about creating more time in the class to be able to cover more 
material. . .having better discussions and getting students really involved in in a deeper layer of conversation” (F4). 

F4 did not think time to be a disadvantage with the flipped classroom model, but a challenge, specifically 
with the belief that more time would be freed up for the instructor and the GTAs to prepare activities and to evaluate 
each student. (F4) 

Overall, instructors believed that while preparation for using the flipped classroom model actually 
consumed more time than for the traditional lecture passive classroom that it was worth it, because the created 
materials could be used for future classes. They also believed that they would not have to spend so much time in 
future class development due to the experience gained with initial flipping preparation both of videos and in-class 
activities. 

Support 


All of the participants indicated that they had some support to use the flipped classroom model in their 
courses. This support came in the form of either, (a) online resources about the flipped classroom; or, (b) other 
instructors’ peer assistance. 

Some instructors received good ideas from Internet resources, such as the online posts about other 
instructors’ teaching experience in the flipped classroom model on wikis and blogs. These ideas were modified for 
use in their own classrooms. Other instructors read journals on instructions of their fields, especially the papers on 
active learning with the same subject matter in the classroom. Even though some instructors commented that reading 
could help to provide some basic ideas, they indicated that peer assistance from other instructors was more useful 
and important to them. 

F 1 and PF2 participated diverse disciplines pedagogy groups. In these groups, they shared and read articles 
about teaching. These groups regularly met to discuss teaching, share ideas and challenges, and help each other to 
solve problems about teaching. 

PF3, F3, and F4 asked the colleagues in their departments for suggestions and support because they felt 
more comfortable speaking with people in their fields. PFl indicated very good collaboration with GTAs on 
instructional design as well. PF2 was more confortable to ask support from a nephew and a sister who both used the 
flipped classroom model in K-12 schools. 

All instructors participated in the same Summer Teaching Institute sponsored by the Teaching and 
Learning Center at UTK. They commented that the workshops and counseling services provided them with great 
ideas in teaching. They felt, however, that what was even more valuable to them was the opportunity for peer 
communication, reflection, and learning from each others about the flipped classroom model provided them by the 
Summer Teaching Institute. 


Discussion 

Research has shown that the flipped classroom model can improve students’ learning motivations, higher- 
order thinking skills, and problem-solving skills. The flipped classroom model can provide students opportunities 
for engagement in student-centered active-learning experiences. Warter-Perez and Dong (2012) state that the 
fundamental idea behind the flipped classroom is that more class time can be devoted to active learning and that the 
instructor can provide immediate feedback during the active learning session. All interviewees in this study highly 
praised the in-class learning activities’ role in motivating students to apply the knowledge they gained. This study 
indicates that instructors viewed motivating students to prepare for the in-class activities a most essential component 
of the flipped classroom model. This study also demonstrates that instructors perceived how to motivate students get 
engaged in pre-class learning as a major challenge in implementing the flipped classroom model in instruction. Most 
current research is on the investigation of the in-class activities, and there is little research on the pre-class learning 
in the flipped classroom model. This study highlights the importance of the effectiveness of the pre-class learning 
activities. 

Current research on the flipped classroom centers on the instractional benefit of the flipped classroom 
model through students’ perspectives. Findings from the current research shows students’ attitudes toward their 
learning experiences in the flipped classroom courses, and their learning gains. In contrast, the present study centers 
on training of instructors and their observations of the critical instructional design connected with flipping. In 
interviews, instructors were not only concerned with the instructional performance of the flipped classroom, but also 
on how they would effectively prepare and use this model in a time-saving and effort-saving way. This study 
indicates the need for helping students to be prepared before class and for the provision of proper motivation toward 


150 



that goal. Instructors must be well organized in their instructional goals and design, must be able to think like 
students (viewing objectives from their perspective). They must also provide students with timely support in class. 
Instructors believe it takes large quantities of time and energy to prepare for the flipped classroom model for the first 
time. However, they believe it is probable that less time and energy would be required in future classes because they 
had already gained both learning materials and instructional experience. Ultimately, support and training of 
instructors on how to use the flipped classroom model efficiently will prove critical to the acceptance of this model. 
Future research should focus on instructor’s support and training in adopting the flipped classroom model. 

In the investigation into the support instructors had for using the flipped classroom model, this study revealed 
that peer communication and assistance between instmctors, such as peer assistance among pedagogy groups, and 
communication with experienced colleagues, were perceived by instmctors as an efficient support approach for helping 
them to use the flipped class by the instmctors. Based on this, future training may need to provide opportunities for 
instmctors to have peer assistance, and support in a more flexible and more confortable way. 

A limitation of this study is that all the participants were from the same university and had the training in the 
same Summer Teaching Workshop, so their backgrounds were similar. Future studies on training for using the flipped 
classroom model need to draw upon instmctors from more diverse backgrounds. Future research might investigate how 
universities can better support faculty members in gaining knowledge and skills required for building more active, 
effective, and multi-modal learning environments and a faculty peer-assisted learning community. 
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Introduction 

Approximately one in six young people has a mental health problem that is severe enough to impair their 
funetioning at home or at sehool and with peers or in the eommunity (Waddell & Shepherd, 2002). Moreover, 
almost half of young people who are dealing with a mental illness have more than one disorder (Sehool Mental 
Health ASSIST, 2012). It is eritieally important for youth who are dealing with mental illness to get help given that 
mental illness is the most signifieant risk faetor for suieide (Weir, 2001) and that suieide the seeond most eommon 
eause of death for Canadians aged 16 to 24 (Navaneelan, 2012). However, a number of eommon myths about mental 
illness ean make them less likely to seek help. One eommon myth is that mental illness is the “fault” of youths’ 
parents or of the young people themselves - who may be eharacterized as attention-seekers. The belief is that they 
eould overeome it with the proper diseipline or suffieient foree of will. A related fallaey is that nothing ean be done 
to treat mental illness. These two misconeeptions are part of the reason why only one in five Canadian ehildren who 
need help with mental health issues receive it (Canadian Mental Health Association, 2012). 

In the context of mental health, the term stigma refers to an extra burden carried by people who suffer from 
mental illness, that is caused by how other people see them (and, in many cases, how they see themselves). Stigma 
has been identified as having as much of a negative effect on youth affected by mental illness as the illnesses 
themselves (Mental Health Commission of Canada, 2010). It has been observed that stigma exerts a particularly 
powerful effect in schools, where youth may fear teasing, bullying and ostracism if they come forward with mental 
health issues. As a result, stigma can make mental illness invisible and may force those who are dealing with it to do 
so alone (Canadian Mental Health Association, 2008). 

The purpose of this project was to conduct an evaluation of the pilot program "Let’s Talk: Finding Reliable 
Mental Health Information and Resources", which consists of both teacher training and the implementation of three 
specific lesson plans for students in Grades 7-8. Given considerable variability in definitions of program evaluation 
(Patton, 2008; Rossi, Lipsey & Freeman, 2004), for the purposes of this paper, evaluation was defined as a 
systematic inquiry leading to judgments about program merit, worth and significance, and support for program 
decision-making and knowledge production (Cousins, 2003; Weaver & Cousins, 2004). Though broad, this 
definition accommodates both summative, judgment-oriented evaluation questions as well as formative, 
improvement-oriented questions. This conceptualization clearly situates the evaluation as a systematic process 
guided by inquiry methods common to the social sciences. The results of the evaluation will be used to guide the 
improvement of the program prior to a national expansion of the program in 2015. 

The evaluation of this pilot program is designed to identify any areas that need improvement and 
development. It focuses on both program process and delivery of program content, and short and intermediate term 
outcomes for teachers and students. For this reason, the current evaluation is formative, meaning that it is designed 
to provide feedback for improving the program, particularly as it pertains to the lesson content, the delivery of the 
program and the online framework for teachers. In this specific paper, we present results pertaining to student’s 
accumulation of knowledge on mental health, and mental health resources as a result of participating in the lessons 
and using their multi-media platforms. 
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Program background 

The bilingual program "Let’s Talk: Finding Reliable Mental Flealth Information and Resources" was 
designed to enable teachers to connect mental health issues to the wider curriculum through media literacy and to 
help students recognize, find and authenticate reliable mental health information and resources online, and by using 
a variety of multi-media tools. The program is comprised of online training for teachers as well as classroom 
resources. Teacher training resources include a self-directed presentation and a teachers’ guide. Using an online 
platform for this purpose allowed for more fullsome participation in a variety of Canadian cities. Classroom 
resources includes six teaching lessons that use a variety of multimedia tools that last an approximate duration of 
one class period for Grades 7-8 (three in English and three in French): 

o Lesson One: Authenticating Mental Health Information Online 

o Lesson Two: Setting the Record Straight: Public Service Announcements on Mental Health and Illness 

o Lesson Three: Social Support Networks and Help-Seeking 

In each of the three lessons in this program, students view a powerpoint presentation about mental health 
through which they are introduced to holistic and universalizing concepts, such as the mental health continuum. In 
Lesson One, using video students evaluate common misconceptions about mental illness and use reliable 
information that is provided to correct those misconceptions by watching and learning about public service 
announcements through online videos and YouTube. Students then have a chance to create their own using film or 
traditional acting. In Lesson Two, students learn media literacy skills by evaluating and authenticating content and 
websites on the Internet in order to locate reliable online information about mental health. In Lesson Three, students 
are guided to recognize and locate different mental health resources available to them within their communities. It is 
here where they use telephone and skype to access a help-seeking hotline. Kids Flelp Phone where they get to talk to 
a counsellor and ask questions during a live chat. 

The program’s learning objectives are as follows: 1) to encourage students to discover reliable public 
information about mental illnesses, which challenges common misconceptions and raises awareness about stigma; 2) 
to increase student awareness of social support networks and formal mental health resources found within their 
communities which should support help-seeking behavior; and finally, 3) to assist students in developing the 
necessary skills to locate and evaluate reliable information about mental health on the Internet, which in turn should 
help to increase awareness about mental health. 


Research questions 

1. To what extent has students’ ability to understand media’s influence on their understanding of, and attitude 
towards, mental health increased? 

2. To what extent has the program enabled students to identify support networks available to them in the 
community and enabled them to cultivate help-seeking behaviours for mental health issues? 

3. To what extent has the program enabled students to be able to find authoritative sources of mental health 
information on the Internet? 


Methods 

The methodology for the evaluation of the pilot program was based on a combination of teacher and 
student participation and was conducted in three phases. 

Phase One: Evaluation Planning (January-June 2013) 

This phase involved a thorough review of all relevant documentation for this program as well as a 
comprehensive review of the literature on program evaluations that are similar in scope, in context and in program 
content. This phase provided an understanding of the "Let’s Talk: Finding Reliable Mental Flealth Information and 
Resources" program and led to a detailed evaluation plan. This plan included a sampling strategy to identify 
approximately 15 schools in 2 provinces (three school boards in the provinces of Quebec and Ontario respectively) 
and then to further select Grades 7 and 8 teachers or classrooms within those schools. Requests from school boards 
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to conduct this research were submitted in Ontario and Quebec in the late spring of 2013. Once school board 
approval was received, the Board sent an open invitation to all schools. Three school boards in Ontario and one 
independent school in Quebec agreed to participate in the evaluation of the pilot program and one of the three 
lessons was implemented in each school board based on a first come first serve basis: 

1. CSDCEO (conseil scolaire catholique de Test de TOntario): Lesson 2 “Setting the Record Straight: Public 
Service Announcements on Mental Illness”; 

2. OCDSB (Ottawa Carleton district school board): Lesson 1 “Setting the Record Straight: Authenticating 
Mental Health Information Online”; 

3. TDSB (Toronto district school board); Lesson 3 on “Social Support Networks and Help Seeking”; 

4. External St. Jean Eudes (independent school in the province of Quebec); Lesson 3 on “Social Support 
Networks and Help Seeking”. 

Phase Two: Instrument Development and Data Collection (September 2013 -June 2014). 

Student learning was assessed via questionnaires with one pre-test (pre-lesson) as well as two post-tests 
(post-lesson) that were administered by participant teachers in the classrooms to assess students’ acquisition of new 
skills and knowledge. The questionnaires were specific to each lesson. The first post-test was administered 
immediately after the lesson; the second post -test or post-post-test was administered two to three months following 
the lesson, in December 2013-January 2014. Codes comprising their mothers’ phone number and name were 
assigned to the questionnaires and consent forms ensuring anonymity. This was for comparative purposes and for 
monitoring changes in desired behaviours and perceptions. 

Teacher feedback was also considered by using an online survey and one-on-one interviews. While that 
aspect of this project is beyond the scope of this article, a few relevant findings from the interviews, in the form of 
quotations will be shared as they relate to student experiences of the multi-media platforms in the results’ section 
followed by a more detailed analysis in the discussion section. 

Phase 3 Putting it All Together (June 2014- August 2014) 

This phase involved the analysis and synthesis of quantitative and qualitative data. The following types of 
analyses were used: 

1 . Repeated measures analyses (quantitative) 

2. Paired sample t-tests (quantitative) 

3 . Thematic coding (qualitative) 

Participants 

Overall 669 grade 7 and 8 students from three Ontario school boards and one independent school board in 
Quebec participated in the evaluation. Parental consent as well as completion of all three questionnaires was 
obtained for 325 students. Therefore analyses and results are therefore based solely on these 325 students. The 
following is a specific breakdown of student participation by school board: 

• CSDCEO (conseil scolaire catholique de Test de I’Ontario): 107 students fully completed the 
evaluation with required consent; 

• OCDSB (Ottawa Carleton district school board): 99 students fully completed the evaluation with 
required consent; 

• TDSB (Toronto District School Board): 91 students fully complefed the evaluation with required 
consent; 

• Extemat St. Jean Eudes (independent school in Quebec): 28 students fully completed the 
evaluation with required consent. 

For the qualitative aspect of this study, 10 teachers across three school boards, excluding the independent school in 
Quebec, participated in one-on-one interviews about their training experiences and their perceptions of their 
students’ learning. 
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Results 


Research questions and corresponding results are presented by school board. 

1. CSDCEO (conseil scolaire catholique de Test de TOntario) 

Teachers in the CSDCEO were instructed to teach Lesson Two: “Setting the Record Straight: Public Service 
Announcements on Mental Illness”. 

Research question #1 

To what extent has students’ ability to understand media’s influence on their understanding of, and attitude towards 
mental health increased? 

Significant differences at the p<0.05 level were detected in a repeated measures analysis regarding 
students’ understanding of advertising between the three assessment periods: pre-test, post-test and post-post-test. 
Next, paired samples t-test between each testing period showed that all three levels of understanding proved to be 
significantly different from one another, indicating that students’ understanding of advertising increased between the 
pre-test and the post-test (p<0.05) and three months later (p<0.001). Moreover, there was a statistically significant 
difference between the post-test and the post-post-test (p<0.001). 

Significant differences regarding students’ understanding of mental illness between the three test 
periods: pre-test, post-test and post-post-test (2-3 months after) were noticed (p<0.001). Paired samples t-tests 
between each testing period revealed that two levels of understanding proved to be statistically significant, 
indicating that students understanding of mental illness increased between the pre-test and the post-test (p<0.000) 
and the post-post-test (p<0.000). 

Teachers also expressed how the online access of the quality of this lesson and materials and video 
component made for a favourable and interactive learning experience for their students. 

“Le fait que c ’est accessible en ligne, ga c ’etait tres gagnant, le fait qu ’il y a avait les presentations a / ’ecran ou 
c ’etait pas juste lire un article. C ’etait tres interactif ga a ete tres gagnant. C ’est quelque chose qu ’Us ont 
I ’habitude de faire a la maison. Dejd la, ga donnait un certain credibilite au niveau de I ’attention, je vais ecouter 
parce que c ’est youtube R 

Teacher 8 

“I taught all three lessons and their favourite was the PSA. They really enjoyed the whole PSA experience. In the 
second one that we did, some of the kids wanted to turn it on its head and do anti PSA ’s because we ’ve been talking 
about technology and that kind of thing. So they were talking mental health issues and seeing it completely 
backwards. And the kids really enjoy that because they found when they did that, it was more explicit for them and 
they were able to talk about it. ” 

Teacher 4 

“We did all the PSAs on ipads. They made imovies and then they edited them and did a really good job! They even 
presented their product to their health teachers and had a great response! They enjoyed the experience and sharing 
and it was fun to watch too!. ’’ 

Teacher 4 


2. OCDSB (Ottawa Carleton District School Board) 

Teachers in the Ottawa Carleton District School Board (OCDSB) were instructed to teach Lesson One: 
“Setting the Record Straight: Authenticating Mental Health Information Online”. 

Research question #2 

To what extent has the program enabled students to identify support networks available to them in the community 
and to cultivate help-seeking behaviours for mental health issues? 
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A repeated measures analysis was statistically significant at the p<0.005 level regarding students 
understanding of the Internet as a research tool. Paired sample t-tests between each and every testing period 
indicated a statistically significant difference between the pre-test and the post-post-test (p<0.005) and a trend 
between the post-test and the post-post-test (p<0.051) regarding students’ understanding of the Internet as a 
research tool. 

Research question #3 

To what extent has the program enabled students to be able to find authoritative sources of mental health 
information on the Internet? 

A repeated measures analysis revealed statistically significant differences at the p<0.05 level regarding 
students’ online search skills. Paired sample t-tests between each and every testing period revealed a statistically 
significant difference between the post-test and the post-post-test (p<0.05) and a trend between the pre-test and the 
post-post-test (p<0.054) regarding students’ online search skills. 

The following quote depicts how students were able to learn from the variety of websites they accessed and 
their experiences in this lesson: 

“the idea of searching the web was kind of an eye opener for them. Sites that they thought would have been 
legitimate before were no longer legitimate. Because they didn ’t have an official, like someone promoting like that 
kind of thing. There was no Dr. Oz, it was just some person writing something that they thought. So that for me, was 
the best thing and I felt that it worked the best because they were actually able to get in there and look things up and 
it allowed them to be more open with each other. It also helped bring a better classroom climate for my students 
because they are learning in the age of technology.” 

Teacher 5 


3. TDSB (Toronto district school board) 

Teachers in the Toronto District School Board (TDSB) were instructed to teach Lesson Three: “Social Support 
Networks and Help-Seeking”. 

Research question #2 

To what extent has the program enabled students to identify support networks available to them in the community 
and to cultivate help-seeking behaviours for mental health issues? 

A repeated measures analysis showed statistically significant differences at the p<0.01 level regarding 
students’ understanding of mental illness. Paired sample t-tests between each and every testing period 
demonstrated a statistically significant difference between the pre-test and the post-test (p<0.012) and between the 
pre-test and the post-post-test (p<0.008; 2-3 months after the lesson) regarding students’ understanding of mental 
illness. 

A repeated measures analysis was statistically significant at the p<0.000 level regarding students’ 
knowledge of Kids Help Phone. Paired sample t-tests between each and every testing period revealed a statistically 
significant difference between the pre-test and the post-test (p<0.000), between the pre-test and the post-post-test 
(p<0.005) and between the post- and the post-post-test (p<0.000) regarding students’ knowledge of Kids Help 
Phone. 

The following teacher quotes depict how students appreciated the KHP call opportunity. 

Highlights: 

“The Kids Help Phone call was incredible. To me that was the highlight. When the date was set I went in the day 
before and told them they were gong to have the opportunity to write questions that they wanted to ask on slips and I 
was going to ask a couple of volunteers to act as ambassadors to read or ask of the Kids Help Phone person. I then 
went through the slips and grouped them by similarities so as to avoid duplications” . 

Teacher 2 
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“/ thought going in to ask them, when I gave them the option to ask questions, and then going in and giving them the 
paper to ask questions, truly thought I would only get 5. I got like 50! Yeah I really thought I would only get a 
handful but the response was unbelievable and students were happy to be interactive Y 

Teacher 1 


4. External Saint Jean Endes 

The sole Quebecois school that participated in the program evaluation taught Lesson Three: “Seeking 
Help”. The following questions examine how students responded to the lesson regarding their understanding of 
mental illness, awareness of mental health resources and their knowledge of Kids Help Phone. 

Research question #2 

To what extent has the program enabled students to identify support networks available to them in the community 
and to cultivate help-seeking behaviours for mental health issues? 

A repeated measures analysis showed statistically significant differences at the p<0.05 level regarding 
students’ understanding of mental illness. Paired sample t-tests between each and every testing period revealed a 
statistically significant difference between the pre-test and the post-test (p<0.05) and between the pre-test and the 
post-post-test (p<0.05) regarding students’ understanding of mental illness. 

A repeated measures analysis indicated statistically significant differences at the p<0.01 level regarding 
students’ knowledge of Kids Help Phone. Paired sample t-tests between each and every testing period showed a 
statistically significant difference between the pre-test and the post-test (p<0.002), between the post-test and the 
post-post-test (p<0.005) and between the post- and the post-post-test (p<0.05) regarding students’ knowledge of 
Kids Help Phone. 


Discussion 

Overall, this paper aimed to evaluate if student’s understanding of mental health problems, resources, and 
media involvement increased from the pilot program "Let’s Talk: Finding Reliable Mental Health Information and 
Resources" and it’s multi-media platforms. Each of the program’s three lessons were successful in enhancing 
students’ levels of mental health awareness. The following paragraphs will discuss the improvements associated 
with each lesson more closely. 

Starting off. Lesson One, which was taught in the OCDSB, will be considered. The evaluation of Lesson 
One focused on students’ understanding of the Internet as a research tool. Results proved that there was a significant 
increase between the pre-test and the post-post-test signaling positive change in how students authenticate mental 
health knowledge online. There was also a significant trend in their understanding of Internet as a research tool 
between the post-test and the post-post-test, suggesting that the impact of the lesson has potential to grow over time. 
This is important knowing how much time youth spend on the Internet and how it can be used both constructively 
and destructively. Students’ understanding of the Internet as a research tool is therefore promising as this could 
enable them to use the web as a constructive tool when identifying support networks available to them in the 
immediate community and perhaps other international resources. 

One of the teachers who taught Lesson One felt that students were able to closely identify with the learning 
and it’s exercise given that it was taking place in an online environment, a forum in which students are comfortable 
and well versed. This teacher found the experience to be an enlightening one for students as they were able to learn 
and appreciate the difference between tmstworthy and untrustworthy online sources of information in their daily 
practice of web surfing. 

Next, student learning in Lesson Two was assessed for the impact of advertising media on students’ mental 
health knowledge (CSDCEO school board). Analyses show significant differences for the three testing periods (pre, 
post, post-post) suggesting improvement in students’ mental health knowledge. Feedback from teachers made a link 
between this positive feedback and the video components oft this lesson. More specifically, they expressed that 
watching PSA’s through YouTube gave the lesson materials and the program added credibility. Students were more 
encouraged and motivated to watch given that the PSA’s came from a web platform with which they were very 
familiar and comfortable. Students also thrived as their creative ideas and acting skills soared in the creation of their 
own PSAs using I Pads and some traditional skits. It allowed them to express themselves individually and amongst 
their peers and put the knowledge they acquired in to practice. 


157 



Lastly, the students from TDSB and the independent sehool in Quebec (Extemat Saint Jean Eudes), learned 
about help-seeking through Lesson Three. Their knowledge and awareness of Kids Help Phone showed significant 
gains. However, one may caution that students’ knowledge of this resource may not necessarily convert into its 
actual use because descriptive analyses showed that students would use this resource sparingly. That said, one may 
believe that “knowledge is power” because these students are more aware of this helpful resource which makes this 
tool relevant. As briefly touched upon in the results section of this article, teachers felt that the live phone call was 
truly a highlight in that it engaged students to actively write down questions, nominate ambassadors of the class to 
participate on the call, and access a live counselor in real time, a credible resource to whom they could ask all of 
their questions. 

Overall, students from three school boards (CSDCEO, TDSB, Extemat St Jean Eudes) significantly 
increased their understanding of mental illness such as definitions of anxiety and depression between the three 
testing points. These results are therefore relevant to consider when launching a national program of this kind, given 
that students were shown to become more cognizant and competent regarding key concepts and issues related to 
media products and mental health. 


Conclusion 

In conclusion, the lesson material in the bilingual program “Let's Talk: Finding Reliable Mental Health 
Information and Resources” was proven to be appropriate and efficacious for its student audience in all three 
Ontarian School Boards and the one independent school in Quebec. The results overwhelmingly affirm that the 
lesson materials and its use of multi-media platforms increased grades 7 and 8 student knowledge with regards to 
better understanding where to turn to and which sources to listen to when it comes to mental health. Media Smarts, 
having designed Let’s Talk program lesson activities in conjunction with variety of different technologies, mainly 
video, the internet, and telephone was able to properly engage middle-school students and instill valuable 
information in their minds. The lessons are congruent with the recommendations of Canadian Mental Health 
Association and the Mental Health Commission of Canada who state that educational materials on mental health that 
engage the audience on a personal, emotional, and intellectual level have proven to be effective stigma reduction 
tools, as they increase favourable attitudes and decreases perceived dangerousness (Canadian Electronic Library, 
2013). 
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Abstract 

Recently, interest in touch-based tablet devices and collaborative learning has rapidly expanded around the 
world. This paper presents a conceptual framework that blends cognitive load theory and collaborative learning 
theory to discuss the impact of tablets on learning discourses. Research indicates using tablets requires fewer mental 
resources when compared to traditional computers. The disconnect between physical action and visual stimuli due to 
a keyboard and mouse is more mentally taxing than a gesture based interface. This extraneous mental effort can be 
detrimental to collaborative learning, where learners must also contend with the coordination, communication, and 
recombination of information. This paper discusses the potential to improve collaborative learning discourses by 
enabling direct interaction with learning content using touch-based tablet devices. 

Introduction 

Touch-based tablet devices, such as the Apple iPad, have gained significant traction and enthusiasm 
amongst educators, students, and technology leaders alike. Billions of dollars have been invested, and even entire 
schools have been created around tablets, yet little is known about how they are impacting learning (Evers, 2013; 
Mathis, 2013). These devices offer novel ways to teach, learn, communicate, and interact with digital content. In 
classrooms they are being used for a variety of learning tasks, including student collaboration (Simpson, Walsh, 

& Rowsell, 2013). The difficulty facing educators is the limited scholarly pedagogical supports on how tablets 
affect student learning. This paper explores available literature to develop a conceptual framework to describe 
collaborative learning with tablets through the lens of cognitive load theory. 

Significance 

The focus on collaboration was selected due to its longstanding widespread prominence in real world and 
academic settings. Johnson, Adams Becker, Estrada, and Freeman (2014) noted collaborative learning as one of the 
driving forces behind the latest educational trends in social media learning, online learning, hybrid or blended 
learning, and face-to-face learning. In non-academic contexts it is commonplace to work in teams and collaborate on 
tasks. Over time, collaboration has followed the general global shift towards digital formats and mediums evident in 
numerous aspects of learning. 

The modem digital era has fostered an explosion of anytime-anywhere learning using mobile devices. Their 
portability, affordability, and always-connected capabilities have made them a popular choice (Johnson, Smith, 
Willis, & Levine, 2011). Mobile devices have triggered a cultural shift in the ways that people communicate, 
collaborate, leam, and interact. They have enabled great flexibility in synchronous or asynchronous collaboration 
discourses and tasks, without the geographical restrictions of traditional computers. What historically began as a 
choice between small functionally limited cellular devices, and larger more expensive laptops, mobile learners can 
now choose tablets as an effective crossover device (Johnson et ah, 2013). More than simply an enlarged cellular 
phone or a shrunken laptop, tablets are a new dynamic and interactive form factor. Accessibility to the internet and 
social networking applications are often deeply integrated into the devices, leveraging the popularity of social media 
sites such as Facebook or Twitter (Johnson et ah, 2013). The mainstream popularity of social media and a desire to 
shift away traditional Teaming in silos’ models, has contributed to mounting pressure on educational institutions to 
incorporate more social and collaborative learning into education curriculums (Helfand, 2013). In response, some 
institutions have turned to tablets as one way to facilitate mobile and collaborative learning (Alberta Education, 

2011; Cassie & Scott, 2011; Johnson et al., 2013). 
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The use of tablets in education is still largely in the preliminary stages, with limited supporting scholarly 
research available. Initial findings have reported enthusiastic satisfaction using tablets, noting benefits such as 
improved student engagement and the ability to collaborate in novel ways (Hutchison, Beschomer, & 
Schmidt-Crawford, 2012; Rowsell, Saudelli, Scott, & Bishop, 2013). Wang (2012) observed a relatively shallow 
learning curve for young learners using tablets compared to a traditional computer, allowing students to progress 
past the interface and focus on content more quickly. The researcher attributed this to the intuitive interface, which 
allowed students to directly interact with content, and mimic common real world gestures (e.g., swiping, dragging). 
Similarly, Pack (2012) argued that the physical interactions of the human body could have significant effects on 
cognitive processing. In their research, students performed multiplication tasks on tablets by manipulating virtual 
objects, known as virtual manipulatives. When compared to identical tasks on a typical computer, students 
performed tasks more quickly and accurately using tablets. These preliminary studies have found many positive 
results, but primarily focused on initial impressions with no consistent theoretical framework to support the reported 
findings. 

Empirical studies on tablets have yielded mixed results, with some showing little or no difference in 
learning gains (Halliday, 2012), and some showing significant gains (Harmon, 2012). These findings suggest that 
perceived satisfaction or enjoyment might not be a definitive indicator of improved learning gains. As one of the 
cruxes of learning (Clark, Nguyen, & Sweller, 2006; Merrienboer & Sweller, 2005), cognitive load theory may be 
one way to dig deeper into understanding how the affordances of tablets are impacting learning. Moreover, with 
technology mediated collaborative learning as a persistent trend in education, there is a growing demand for 
theoretical supports to inform educators and decision makers on the use of tablets for learning. This paper seeks to 
answer this call by using the lens of cognitive load theory to explain some of the observations noted by researchers. 

Cognitive Load Theory 

Learning can be a challenging endeavour for students, especially concerning complex topics. One approach 
in the consideration of learning is cognitive load theory (CLT) (Clark et ah, 2006; Sweller, Van Merrienboer, & 
Paas, 1998). This theory divides cognitive load into three types: intrinsic, germane (relevant), and extraneous 
(irrelevant). In brief, intrinsic load is a function of the complexity of the learning content, extraneous load 
corresponds to the mental effort required for all tasks not directly related to learning, and germane load corresponds 
to learning. Working memory is used to process the sum of these loads simultaneously, but is limited by a finite 
capacity, and if exceeded can hinder learning (Kalyuga, 2012). Thus, to maximize learning cognitive load needs to 
be managed and kept within the limits of working memory (Kalyuga, 2012; Mayer & Moreno, 2003). 

Sweller (2010) also describes what is known as the modality effect. The underlying principle is that by 
using both auditory and visual processors (modes), working memory can be expanded. This benefit of dual-modality 
can be used to overcome situations where using a single mode exceeds the capacity of working memory. Reid, 
Stmadova, and Cumming (2013) observed that tablets could support dyslexic students, which typically have 
difficulty with language-based learning. Although they did not explicitly refer to the modality effect, they claimed 
that the combined visual and tactile sensory inputs of tablets allowed dyslexic students to learn more efficiently, 
suggesting some correlation with the effect. 

Segal (2011) found that students using tablets reported a reduced mental effort when compared to using 
computers for an identical task. The researcher was able to show that gestural interfaces reduced cognitive load and 
improved learning. Using a mouse interface created a disconnect between physical movements and what was 
visually displayed. Conversely, tablets allowed students to directly interact with content using gestures that 
corresponded to physical movements, which yielded higher student performance. There are characteristic intrinsic 
loads associated with tablets, computers, and the learning content. Given that these are all interacting elements, 
when in use they consume a certain amount of human working memory. In the computer based scenario (of Segal’s 
research), it is conceivable that the extraneous load was higher due to the additional working memory required to 
negotiate the physical/visual disconnect created by the keyboard-mouse interface. Conversely in the tablet scenario, 
the extraneous load may have been lower due to the intuitive use of gestures and direct interaction with content. For 
both cases the intrinsic load associated with the learning content remained unchanged. Returning to the 
dual-modality effect, the expansion of working memory only occurs when both modes directly correspond with 
each other. The two modes (physical and visual) of the mouse interface did not likely correspond, while for the 
tablet touch interface they likely did, possibly creating an expansion of working memory. This expansion would 
have allowed for increased germane load, and in turn improved performance (Sweller, 2010). 

Novice computer users that must actively look at the keyboard and mouse during operation (e.g., children), 
may be especially partial to the extraneous load caused by splitting their attention between the interface and content 
(Sweller, 2010). Schrader and Bastiaens (2012) argued that if learners are able to minimize or even eliminate the 
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conscious acknowledgement of an interface, their attention on content can be sustained and learning may be 
improved. In their research on gaming environments, a high virtual presence (the feeling of being immersed in a 
virtual environment) made learners feel that game tasks were personally relevant to them, and heightened their 
engagement with the game. To realize gains in learning outcomes, they advised that games should be designed such 
that learners are engaged with learning content rather than just the environment or winning the game. Similarly, 
Hwang, Wu, Zhuang, and Huang (2013) attributed efficient access to information as a way to reduce intrinsic 
cognitive load and improve learning. In order to maximize learning, these studies emphasize the need to manage 
various aspects of cognitive load and keep it within the limits of working memory. 

Collaborative Learning 

Collaborative learning (CL) is generally considered beneficial under the assumption that multiple minds 
will be more fruitful than a single mind. Since there are many perspectives on what constitutes as collaborative 
learning, this paper assumes the broad definition offered by Dillenburg (1999): “a situation in which two or more 
people learn or attempt to learn something together” (p. 2). Advocates cite discussion, constructive debate, deeper 
and more meaningful processing of information as some of the typical benefits of collaboration (Kirschner, Paas, & 
Kirschner, 2008). Despite these, researchers note that there are transactional inefficiencies in collaboration that also 
need to be considered. 

The cognitive load that an individual can handle in working memory is finite, and some tasks may 
overwhelm that capacity. Thus, it can be reasonably argued that the larger collective mind of a group would be 
advantageous for complex tasks. In practice, the transactional costs of coordination, communication, and 
recombination of information creates extraneous cognitive load (Kirschner et ak, 2008). The researchers note that 
for simple tasks, it is less efficient for a group to work collaboratively due to the transactional costs. In other words, 
from the perspective of cognitive economies, collaborative learning should be reserved for complex tasks to 
maximize benefits. The caveat of this perspective is that it is targeted at scenarios where the end goal is to complete 
a task in the most efficient way possible. In the case of brainstorming or concept mapping, the goal may instead be 
creativity or alternate perspectives, with efficiency being a lesser or non-existent concern. 

Discourses 

Language and discourses are key elements of collaboration, and are vital identifiers of learning (Gee, 

201 1). As Gee explains, language is underpinned by identity and intent. Without these, language and discourses hold 
no meaning. He describes identity as the role, agency, or perspective, while intent as the goal or objective that a 
person assumes when communicating. In addition, variations in language, vocabulary and intonation can help place 
learners along a knowledge level continuum between novice and expert (Gee, 2011). In a collaboralive selling, 
differences in knowledge level will force learners to elaborate on their knowledge, and through the negotiation and 
justification of information with each other, gain a deeper understanding of the subject matter (Pfister, 2005). The 
authors also highlight that elaborative discourses among students are not usually spontaneous and require systematic 
supports (e.g., technology, pedagogy, direct human intervention or guidance) to improve learning. 

Another perspective is academic discourse socialization, which describes the process of new or novice 
students being assimilated into existing academic discourses through social interactions (Marriott, Moore, & 
Spence-Brown, 2007). Observing and analyzing such interactions could allow for the identification of knowledge 
transfer and exchange. Ploetzner, Dillenbourg, Preier, and Traum, (1999) similarly describe the concept of 
Teaming by explaining to others,’ as an effective approach to facilitate knowledge construction and deeper 
understanding. This exchange of knowledge works to achieve what is known as a common ground of information, 
where by the end of the interaction participants share a relatively similar understanding of the subject matter (Pfister, 
2005). For example, Simpson, Walsh, and Rowsell (2013) observed an interaction between two elementary school 
students working on separate tablets to complete a reading task together. In the study, students used the tablet’s 
‘pinch-tozoom’ to emphasize areas of interest on the screen to each other, and simultaneously used both oral and 
gestural interactions to collaboratively complete the task. Also in this instance, one student was more comfortable 
using the tablet and was able to navigate to a specific webpage more quickly than their partner. The partner, unsure 
of how to access the webpage, observed the more experienced student to guide their own actions. The bulk of the 
interaction was non-verbal and demonstrated one of the alternative ways that students can learn and interact with 
each other. 

Tangibles 

Interfaces embedded with some degree of computational power are known as tangibles (Price, 2013) or 
digital manipulatives (Manches & O’Malley, 2011). These technologies can take many different forms such as cubes 
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embedded with microchips (Manches & O’Malley, 2011), interactive multitouch surfaces (e.g., tablets, tabletops), or 
LightTables which use light beams and digitally enhanced blocks (Price & Jewitt, 2013). Tangibles offer novel ways 
for learners to interact with digital learning content. For example, interactive tabletops have been explored as a tool 
for facilitating collaborative learning (Battocchi et ah, 2010; Morris, Fisher, & Wigdor, 2010; Schubert, Serna, & 
George, 2012; Stahl, 2011; Yuill & Rogers, 2012). These are large devices (e.g., forty or more inches diagonally) 
with multitouch surfaces, where multiple users may interact with the surface simultaneously. One of the advantages 
of these devices is that users are not restricted to a single point of control as with a mouse-based computer interface, 
where the physicality computers can make collaborating more cumbersome. Tabletops reduce or eliminate the need 
to explicitly negotiate which user controls the device at any given time (Fliggins, Mercier, Burd, & Flatch, 2011). 
This negotiation could be classified as an extraneous load that is irrelevant to learning but necessary in a 
collaborative context (assuming group members alternate control of the computer). Interestingly, Do-Lenh, Kaplan, 
and Dillenbourg (2009) found that while students preferred using a tabletop interface, they actually performed better 
using a computer. They attributed this to the forced need for students to negotiate and communicate which student 
controlled the computer, due to its single point of control. Moreover, the physical constrictions prompted them to 
negotiate specific tasks and roles for students not controlling the computer. This may imply the poor performance 
was actually due to the poor organization of roles and task assignment, rather than the interface. 

Martinez-Maldonado, Dimitriadis, Martinez-Mones, Kay, and Yacef (2013) studied verbal and physical 
interactions of learners collaboratively using a tabletop. Their findings showed that when there were high levels of 
simultaneous uncoordinated verbal and physical interaction, the quality of collaboration was low. Conversely, 
when students alternated between high levels of coordinated discussion and physical interaction, the quality of 
collaboration was high. Additionally, when one student orally accompanied or supported the physical actions of 
another student, collaboration quality was also high. The results of Do-Lenh et al. (2009) and Martinez-Maldonado 
et al. (2013), indicate that the benefits of a tabletop are most likely to be realized when there is a mutual 
understanding between users and their efforts are coordinated in unison. A similar approach could be taken with 
tablets to ensure that they contribute to learning rather than group dissonance. 

Implications of Literature 

The surveyed literature indicates that educators are enthusiastic on using tablets for learning. Initial 
findings are positive with increased engagement, portability, affordability, intuitive interface, and general 
satisfaction commonly cited as benefits. Empirical studies have found mixed results, with some showing significant 
improvement in student performance, while others showing little or no improvement. The principles of CLT suggest 
that tablet interfaces can have a significant impact on student learning. A combination of gestures, intuitive design, 
and the ability to directly interact with content all affect cognitive load. These effects are further emphasized during 
collaboration, when learners must also negotiate discourses and interactions. These theoretical perspectives provide 
insight into some of the ways that tablet interfaces are impacting learning and the ongoing cultural shift towards the 
increased use of mobile devices. The literature also reveals that further research is required to develop pedagogical 
supports to inform educators. 


Framework 

This framework proposes to explain how a multitouch tablet interface can impact collaborative learning 
from the perspective of cognitive load. Tablets offer unique affordances not readily available with other digital 
devices. They allow learners to directly interact with content using gestures, omitting the need for an intermediary 
input device (e.g. mouse and keyboard). In turn this intuitive interface may affect learning. Using a mouse and 
keyboard with a traditional computer requires a certain amount of mental effort. Users must determine which keys to 
press and where to move the mouse to achieve a task, all of which requires mental effort. For novice users 
negotiating these input devices can be especially distracting and detrimental to the completion of the task at hand 
(Clark et al., 2006), and even experienced users must expend some mental effort. In addition, there is an inherent 
disconnect between the user’s physical actions and what is visually displayed. The mental effort required to use a 
keyboard and mouse can be classified as extraneous cognitive load (Sweller, 2010). For learners, this type of 
cognitive load does not directly contribute to learning but is necessary to complete a task using a computer. This 
extraneous load can hinder learning by forcing students to alternate their attention between the learning content and 
the input devices (Baker, 1999; Sweller, 2010). Under CLT learners must divide their limited working memory 
among three types of cognitive load: intrinsic, germane, and extraneous. Where intrinsic is fixed as a function of the 
complexity of the learning content, germane is used for learning, and extraneous is anything not directly related to 
learning. 
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The advantage of using a tablet eompared to a eomputer is that students are able to reduee their extraneous 
load (Paek, 2012). As a result, students ean learn more effeetively by alloeating more working memory to germane 
load. This reduetion in extraneous load is due to the omission of intermediary input deviees, allowing students to 
direetly manipulate eontent. The benefit is further eompounded when visual and kinesthetie stimuli eorrespond to 
eaeh other, leading to the modality effeet (Sweller, 2010). When this oeeurs working memory expands, allowing 
learners to negotiate tasks of greater eomplexity. Additionally, toueh interfaees ean faeilitate the use of gestures sueh 
as swiping or dragging, whieh ean reduee eognitive load and improve performanee (Goldin-Meadow, Nusbaum, 
Kelly, & Wagner, 2001; Segal, 2011). 

Collaboration 

In eollaboration, students work together to aeeomplish a learning task. This ean oeeur in many different forms with 
and without teehnology. For simplieity, the following diseussion is orientated around a seenario where two 
learners share a single tablet. To eomplete a learning task, students must use working memory to proeess the 
eorresponding eognitive load from three primary sourees: interaeting with the tablet, interaeting with the other 
student, and engaging with learning eontent (see Figure 1). The intrinsie eognitive load of interaetions with the 
tablet and of the learning eontent is assumed to be largely independent of eollaboration. This is beeause for a given 
task, the tablet interfaee and the learning eontent is eonstant, and does not beeome more eomplex when 
eollaborating. Conversely, interaetions with the other student ean eontribute multiple levels of eomplexity to the 
overall learning aetivity. 

The interaetions between the two students ean be divided into three main eategories (see Figure 1): verbal 
diseourses, non-verbal diseourses, and diseourses supported (mediated or augmented) by the tablet. Typieal verbal 
diseourses ean inelude negotiation and justifieation of information, elaboration (Pfister, 2005), explaining oneself 
to others (Ploetzner et al., 1999), general diseussion, and eonstmetive debate (Kirsehner et al., 2008). Non-verbal 
diseourses ean inelude hand gestures while speaking, faeial expression, gaze, and physieal posture relative to the 
other student and the tablet (Gee, 2011). Diseourses mediated or augmented by the tablet ean inelude pointing to a 
point of interest while speaking, writing or drawing diagrams on the tablet as part of eommunieation, and 
manipulating digital eontent to emphasize points of interest (e.g., ‘pineh-to-zoom’ or panning) or show alternate 
perspeetives (e.g., rotating a three dimensional objeet) (Simpson et al., 2013). All of these interaetions eontribute to 
the overall eognitive load of the learning aetivity that must be negotiated by students. In seenarios involving highly 
eomplex tasks or more individuals, the eognitive load of interaetions ean be even higher. Thus, the eognitive load 
assoeiated with using a digital deviee to support student interaetions should be earefully eonsidered so as not to 
overload the working memory of learners. 

Another important faetor in using digital deviees for eollaboration is the interfaee. A eumbersome interfaee 
(e.g., keyboard-mouse) ean eause students to split their attention between the deviee and other students, making 
learning and interaetions more diffieult. Conversely, an intuitive interfaee (e.g., toueh sereen tablet) is able to fade 
into the baekground of students minds, allowing them to beeome more engaged with diseourses and learning eontent 
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I Intrinsic | Germane H Extraneous 

Figure 2: Hypothetical example of cognitive load distribution (percentage values are arbitrary) when 
comparing students collaboratively completing identical tasks on a computer and tablet. 

(Schrader & Bastiaens, 2012). 

Deeper understanding and meaning making is most readily achieved when learners are able to sustain 
their attention on one thing at a time (Baker, 1999; Sweller, 2010). Compared to computers, tablets have the 
potential to reduce the cognitive load corresponding to the use of a device (see Figure 2). This reduction in 
cognitive load could allow learners to dedicate more working memory to interactions other people and learning 
content. Moreover student discourses could achieve higher quality and greater depth due to the increased 
availability of working memory to process germane load. 

The benefits of using gestures to interact with content can also be extended to collaborative settings. Given 
the principles of the modality effect, it may be possible for students expand working memory by using tablets to 
mediate gesture-supported discourses. An easy to use human centered interface can promote students to 
communicate with the support of external representations more readily (Norman, 1993), and expedite the process of 
achieving a common ground of interaction (Baker, 1999). Developing and maintaining a common ground of 
information, interaction, and task completion strategy are important factors towards achieving high quality 
collaborations (Martinez-Maldonado et al., 2013; Oliveira, Tinoca, & Pereira, 2011). A tablet gesture based 
interface cannot only minimize the cognitive demands of using technology to support collaboration, but also enable 
instinctive discourse interactions. 

Limitations 

Many of the benefits of using tablets to support collaborative learning are dependent on the interface, which 
in turn is significantly dependent on the design of the software application that is used. A poorly designed 
application or simply 
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using an application that is not well suited for a given task can diminish many of the benefits. The careful selection 
of both hardware and software is a critical element in maximizing the advantages of using tablets. 

This framework is situated around a specific scenario (i.e., two students sharing one tablet), and may 
require further considerations to transfer to other situations (e.g., more than two students, multiple devices, 
combinations of virtual and physical shared spaces). That said, the cognitive benefits of tablet interfaces, and 
the ability to use gestures can be applicable to other scenarios. 

Implications and Fntnre Directions 

This framework offers educators one approach towards understanding some of the intricacies of 
collaboration using tablets. Through the lens of cognitive load, learning processes can be examined in-depth and 
paint a clearer picture of how tablets are impacting collaborative learning. Educators can use this knowledge to 
assist in the strategic selection of hardware and software that will be of the greatest benefit to students for 
specific learning tasks. 

The practical validity of this framework has not been verified and is based on a synthesis of established 
theoretical models. A rigorous validation would be the next stage of development, although as other researchers 
have noted the complexities of collaborative learning are challenging to analyze (Martinez-Maldonado et al., 2013; 
Meier, Spada, & Rummel, 2007). 


Conclusion 

The literature and the proposed framework indicate that interactive technologies can reduce cognitive load, 
by allowing students to use gestures coupled with an intuitive interface to interact more directly with learning 
content. This reduction can improve learning and allow students to process information more easily. Regarding 
collaborative learning, this framework identifies three primary sources of cognitive load: tablet use, complexity of 
learning content, and interaction with others. The interactions with others are further broken down into three types: 
verbal discourses, non-verbal discourses, and tablet supported discourses. The relations between these elements all 
affect cognitive load and moreover learning. This paper develops a framework using cognitive load theory to 
describe some of the ways that tablets can impact collaborative learning. Collaborative learning and technology 
continue to be persistent trends in education with no signs of slowing down. To support educators and decision 
makers, theoretical pedagogies must continue to evolve and be relevant to the ongoing rapid developments in 
technology. 
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Abstract 

The aim of this study is to research the effect of self-assessment on achievement in an online course at 
higher education level. The study was conducted with 688 participants (360 male and 328 female) at online Project 
Culture course at a foundation university in Turkey. A one-way between subjects ANOVA was conducted to 
compare the effect of self-assessment on achievement depending on attempt number. Our results suggest that when 
attempts to self-assesment increase achivement level increase as well. 

Introduction 

The concept of self-assessment is not new and is discussed in many instructional design textbooks (Gagne, 
Briggs & Wager, 1988; Gale 1984; Rowntree, 1991; Laurillard, 1993). Self-assessment, the process of 
understanding more about oneself is a valuable skill, both in life and when studying (Taylor, 1998). 

The concept of open and distance learning suggests an educational approach designed to reach learners in 
their homes/offices/shops etc, provide learning resources for them to qualify without attending formal classes in 
person, or create opportunities for lifelong learning, no matter where or when they want to study (Letseka & Pitsoe, 
2012). It is particularly important for distance education students since learners are in remote locations and isolated 
from instructors and other learners. Self-assessment can result in major benefits both for instructors and students 
(McConnell, 2006), specifically it is more oriented to students, provides instant feedback. 

Furthermore, the students become less dependent on their instructors, responsible and autonomous; they 
take on a more proactive role and develop self-confidence (Cukusic, Garaca, Jadricr, 2014). 

The aim of this study is to research the effect of self-assessment on achievement in an online course at 
higher education level. 


Method 

The study was conducted with 688 participants (360 male and 328 female) at online Project Culture course 
at a foundation university in Turkey. 

Table 1: Number of Students in Project Culture Course 


Gender 

Number of Students 

Male 

360 

Female 

328 

Total 

688 


Project Culture course is a foundation course of the university. The main point is to provide the basic 
information for the students to perform practicable projects in the direction of their real experiences and support 
them with producing new projects. 

This course is chosen by the students studying on Bachelor programs (Faculty of Science and Letters, 
Faculty of Fine Arts, Faculty of Law, Faculty of Economics and Administrative Sciences, Faculty of 
Communication, Faculty of Architecture and Faculty of Engineering) or on Associate Degree programs (Vocational 
School). It is a compulsory course for students of all departments within each faculty and vocational school 
program. Number of student enrollment in terms of faculty and vocational school program in Table 2. 
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Table 2: Enrollment by Faculty and Vocational School program 


Faculty 

Number of Studeuts 

Faculty of Science and Letters 

66 

Faculty of Law 

132 

Faculty of Economics and Administrative Sciences 

137 

Faculty of Communication 

60 

Faculty of Architecture 

79 

Faculty of Engineering 

123 

Faculty of Art and Design 

47 

Vocational School 

44 

Total 

688 


Project Culture course is composed of 7 modules. There is a self-assessment quiz at the end of the each 
module. Self-assessment quizzes are designed for distance students in order to provide them with an opportunity to 
ascertain their mastery of particular topics. Due to their advantages the online multiple-choice (MC) questions are 
used. The purpose of self-assessment quizzes is to allow students to assess how much of the unit they understood. A 
self-assessment quiz consists of ten multiple-choice questions concerning issues raised by the related module. The 
quiz questions cover all of the main points of modules. Sakai CLE was used as learning management system. It 
incorporates functionalities for developing and processing online tests. For each module a question pool contain at 
least 100 questions was created. Students come across 10 different random questions at each attempt. After each 
attempt student was received automated feedback. Feedback includes a short explanation about right and wrong 
answers and refers to related topic in module. Number of attempts is not limited. Attending this self-assessment 
activity is voluntary. 

A one-way between subjects ANOVA was conducted to compare the effect of self-assessment on 
achievement depending on attempt number to self-assessment activates in four conditions (Table 3), which are “No 
attempts to self-assessments”, “Attempts to self-assessments of some of the modules”, “One attempt to self- 
assessment of each module”, “Multiple attempts to self-assessment of each module”. 

Table 3: Coding for attempts made to self-as sess ments 


Code 


# of Students 

0 

No attempts to self-assessments 

51 

1 

Attempts self-assessments of some modules 

234 

2 

One attempt to self-assessment of each module 

173 

3 

Multiple attempts to self-assessment of each module 

230 


Total 

688 


In Table 3: 

• “No attempts to self-assessments” refers to participants who did not attempt to self-assessments and took 
none of the quizzes. 

• “Attempts to self-assessments of some of the modules” refers to participants who took the self-assessment 
quiz of some the modules, not all of them. 

• “One attempt to self-assessment of each module” refers to participants who made one attempt to self- 
assessment of each module and took the each self-assessment quiz once. 

• “Multiple attempts to self-assessment of each module” refers to participants who made multiple attempts to 
self-assessment of each module and take the each self-assessment quiz more than once. 
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Results 


First of all there is no signifieant differenee between total eourse grade of female and male students, as seen 
in Table 4: 

Table 4: Achievement of Students in Project Culture Course 


Gender 

N (688) 

Total Course Grade 
Average 

Male 

360 

72,17 

Female 

328 

72,81 



72,48 


Assessment is an important part of learning proeess. It is linked with learning experienee of students. In this 
study, students assess themselves by taking self-assessment quizzes at the end of the eaeh module. 

A one-way between subjeets ANOVA was eondueted to eompare the effeet of self-assessment on 
aehievement depending on attempt number (Table 5). 

Table 5: Effect of self-assessment on achievement depending on attempt number 



Sum of Squares 

df 

Mean Square 

F 

Sig. 

Between Groups 

76382,68 

3 

25460,89 

640,96 

,000 

Within Groups 

27170,51 

684 

39,72 



Total 

103553,19 

687 





There was a signifieant effeet of self-assessment attempts on aehievement at the p<.05 level for the four 
eonditions [F(3, 684) = 640,96, p < 0.001]. 


Table 6: Group summary details 



N 

Mean 

Std. Deviation 

Std. 

Error 

95% Confidence 
Interval for Mean 

Lower 

Bound 

Upper 

Bound 

No attempt 

51 

49,02 

8,36 

1,17 

46,68 

51,37 

Attempts some of the modules 

234 

64,54 

6,18 

,40 

63,74 

65,34 

An attempt for each module 

173 

74,18 

5,94 

,45 

73,29 

75,07 

Multiple attempts 

230 

84,47 

6,16 

,41 

83,67 

85,27 

Total 

688 

72,47 

12,28 

,47 

71,56 

73,40 


Post hoe eomparisons using the Tukey HSD test indieated that the mean seore for partieipants who did not 
attempt to self-assessments (M = 49.01, SD = 8.36) were signifieantly different than partieipants who attempt self- 
assessments. Post-hoe tests also indieated that partieipants who Attempts self-assessments of some modules (M = 
64.54, SD = 6.18) were signifieantly different than partieipants who made One attempt to self-assessment of eaeh 
module (M = 74.17, SD = 5.94) and Multiple attempts to self-assessment of eaeh module (M = 84.46, SD 

= 6.15). 

Speeifieally, our results suggest that when attempts to self-assessment inerease aehievement level inerease 
as well. The development of an assessment strategy should take into aeeount the purpose of the evaluation results 
plus it should always be planned simultaneously with the preparation of learning aetivities. 

During the proeess, assessment ean be pereeived as one of the enablers of innovation and ehange in an 
edueational setting (Cukusic, Garaca, Jadricr, 2014). The selection of assessment techniques and appropriate 
assessment tools is an integral part of planning the e-leaming processes. 
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1. Introduction 


The increasing prevalence of information and communication technologies (ICTs) in the educational 
system of many countries has been accompanied by a proliferation of research documenting their significance for 
teaching and learning (Moos and Azevedo, 2009). “Blended” e-leaming systems that integrate technology in 
classroom learning have been presented as a promising alternative learning approach (Graham, 2006). The emerging 
trend of integrating ICT tools, namely interactive videoconference, email, online chat, and digital and social media, 
in education emphasizes a collaborative construction of information and can support cross-cultural awareness, 
linking school experiences to real-life situations. These mechanisms and learning “spaces” can nurture the 
development of vital cognitive, social, and technical skills critical to the preparation of future citizens for the 
existing and emerging knowledge societies (Anastasiades, 2009). 

This research paper seeks to contribute to an understanding of how international education programs that 
focus on cross-cultural interaction perceive the value, role, and challenges of information and communication 
technology in achieving their goals. The paper explores how different international education programs see the 
capacity of Interactive Videoconferencing (IVC) and chat rooms to assist in achieving their program goals, by 
examining three New York City- based nonprofit organizations. These organizations have similarly articulated 
goals, but their process of utilizing technology is distinct, informed in part by the age of the participants (middle 
school or university students) and the goals of the program (awareness building or altitudinal change in students). 

The aim of this paper is to present the perspectives of key international education programs on the 
methodologies they employ in using ICT to connect students toward achieving goals of cross-cultural understanding. 
This exploration may serve as a valuable contribution to larger conversations on the pedagogical use of technology 
integration in international education. 

The structure of this paper is as follows: The second part describes the key academic literature on the 
research topic. It is followed with a description on the methods employed in undertaking the research. In the fourth 
and the fifth parts, the major findings and their relevance and contribution to the broader conversation in this field 
are discussed. The lasf parf of the paper presents the concluding remarks and provides food for thought for the 
readers. 


2. Literature Review 


According to Bacon and Kischner (2002), the goal of a new global curriculum for the twenty-first century 
must be to broaden students' perspective to the point at which they begin to view themselves as global citizens in a 
rapidly changing world, where they encounter, compare, and adopt multiple international views. Worldwide, there 
has been widespread expansion of technology-based education leading to an ever growing interest in how 
multiculturalism comes into effect (Garcia-Penalvo, 2013). The pedagogical design of interactive videoconference 
(IVC) contributes to the opening up of classrooms to new communities, and fosters familiarization with new 
learning and cultural experiences and alternative and innovative learning approaches (Anastasiades 2009). 

According to Anderson (2008), videoconferencing plays an important role in the networked distance education tool 
set. But as a stand-alone system, it does not provide the level of student engagement with teachers, other students, or 
content, sufficient to sustain their attention, enthusiasm and, ultimately, high levels of learning. As Clark (2000) 
mentioned, it is the application, design, and ways that the technology is used that determines its educational value - 
not the simple acquisition or use of the technology. Newman (2008) states that although the application of IVC and 
chat rooms in schools is increasing significantly, there is a lack of pedagogical design in the educational IVC 
projects. In a survey conducted by one of the studied nonprofit organization, a number of students expressed that 
they would like the length of the videoconferences to be longer than one hour to further delve into the subject 
matter. Furthermore, many students suggested that there should be a maximum of three schools participating in the 
conference at one time. 


172 



This research seeks to study perceptions of the role of technology in achieving cross-cultural understanding 
among students. It is interesting to note that Means (2003) believes that ICT probably holds more potential for 
university education than it does for K-12 school education. The research studies different organizations engaging 
both university and secondary school participants. 

If one believes that IVC is like a conventional “face-to-face instruction,” he/she will be disappointed 
(Anastasiades, 2009). Evaluations of the effectiveness of the educational use of IVC indicate that it has not yet met 
participants’ expectations (Delaney et al., 2004). According to Ferran and Watts (2008), adult attendees of 
videoconferences must work harder to interpret information delivered during a conference than they would if they 
attended face-to-face. Participants do not share the same three-dimensional space, as they can watch only what the 
camera shows, so critical nonverbal contact among participants is often vague. Heamshaw (1998) claims that IVC is 
considered to be beneficial in supporting dialogue, but may not be appropriate for decision-making. Students 
evaluations of their participation in videoconferencing with one of the three nonprofits studied in this research 
echoed that they were not pleased with the impersonal feeling of the videoconferences, coupled with the “ping-pong 
effect” which happens time-to-time in videoconferences. 

It is worth mentioning that the biggest concerns of students in interactive videoconferencing were 
frustrations with schools who were not prepared to engage in dialogue due to their lack of familiarity or comfort 
with the subject matter; frustrations with technological lags, especially when schools in different countries operate 
on different bandwidth speeds; and language/accent challenges, which made the interaction difficult to understand. 

While the educational programs engage groups through the medium of technology, Sim and Bovard (2004) 
believe that the quality of facilitation of discussion amongst the groups is highly correlated to the quality of 
experiences students have in online interaction. Sheingold (2005) comments, “there is a dearth in the research on 
what facilitators do in varied online learning environments or how facilitation contributes to interaction and 
learning.” Anderson et al. (2001) suggest that “effective online discourse must also be guided toward higher levels 
of learning through reflective participation, as well as by challenging assumptions and diagnosing misconceptions.” 
According to these authors, the online facilitator is the person who makes sure that learning happens and he has to 
maintain a teaching presence that in turn allows collaborative knowledge construction. This research explores 
program views of the role of the moderator/facilitator in online dialogues and projects. One of the educational 
nonprofit’s student feedback report (2009) reflects that in terms of the effectiveness of the moderators, students 
generally provided positive feedback, where 37.7% rated the moderators as effective. Nevertheless, 10% of students 
surveyed believed that the moderators were not effective, due to the fact that students did not get enough time to 
speak and interact with youth from other schools. 

Punie (2007) offers a carefully reasoned projection of the future benefits of ICT for education by firmly 
grounding this technology in the changing social realities of the early twenty-first century. In the time when much of 
the world is transitioning from late industrial societies to knowledge-based societies, Punie argues that ICT will 
necessarily be located at the center of what he calls “learning spaces,” where the educational needs of learners will 
have priority. At the same time, however, the guidance of teachers will still be essential. Moreover, these new 
learning spaces will only be able to function properly if they are embedded in a cultural and social context that is 
firmly committed to ongoing innovation in all aspects of life. The author admits that ICT has not lived up to its 
educational promise yet, but he believes that it will do so in the near future. As Unwin (2007) observes, “it is not the 
availability of the technology which is important, but how it is used” (p. 300). 

Considering the existing literature, this research seeks to contribute to the expanding conversation, in areas 
of international education programming, about the value and best approaches to utilizing interactive 
videoconferencing and online chat rooms to facilitate cross-cultural understanding. 
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3. Methods 


Research Question: How do International Education programs that focus on cross-cultural interaction use 
IVC and chat rooms to achieve their goals? 

A validation field study involving three global educational nonprofits was conducted. Multi-source data 
including interviews, surveys, live and archived videoconferences, classroom observations and small group 
discussions were studied. A combination of the case study method of Stake (1995) and the method of constant 
comparison outlined by Glaser and Strauss (1967) was used for data collection. Interviews coding and participants’ 
evaluation were matched to increase the validity of the study. Four sub themes were identified across all the three 
organizations that serve as comparative findings of this qualitative research. 


After the formulation of the research question, one of the three nonprofits was chosen as an observation 
site. Additionally, program director, assistant directors, and CEO of the three respective organizations were 
interviewed. All have direct observation experience, as well as extensive knowledge of the program and its goals 
and outcomes. A variety of difficulties were encountered throughout the methods process. Since it was originally 
difficult to secure an informative observation opportunity that would be relevant to the research question, it was 
concluded that the most realistic option was to choose a videoconference from these organization’s online database. 
However, this form of observation lacked key components of a live observation and hindered the ability to observe 
all aspects of the interactions taking place. Instead, the video darted between four to five schools, and cut out at the 
most opportune times for crucial observation. It was difficult to pull significant conclusions from these observations 
considering the limited opportunities to witness extended dialogue between students. In addition to this, glitches, 
frozen screens, skipping and cutting out, obstructed the ability to observe an uninterrupted flow of dialogue. 

There are also inherent limitations in the organization’s program outcomes that were given by one of the 
organizations. There was only a small portion of schools/university students that responded to the online survey 
conducted by these studied nonprofits. Since this sample size is small in comparison to the number of schools 
which actually participate in the live videoconferences, the results of the survey could be skewed. 

Observing a live videoconferencing session in one of the organization’s office with middle school students 
who were on an educational tour from Dallas, Texas was the first-hand data collection experience. However, this 
videoconference had a number of technical issues that affected the observation. After couple of minutes into the live 
videoconference with Haiti, the office lost signal and the connection could not be restored with the other schools for 
the remainder of the videoconference. An intern did facilitate a discussion regarding the topic to be discussed with 
the group of students who were set up in the nonprofit office. An impromptu interaction was observed, although it 
reinforced the idea that technology comes with its own challenges which might not be controlled even after all the 
preparations for a fruitful student’s interactions via IVC and online chat rooms were already planned. 

4. Findings 

In order to explore how three key international education programs understand the role and value of 
technology in helping to achieve their goals of cross-cultural understanding, it was critical to examine why and how 
they articulate their engagement of technology, and how they perceive its benefits and challenges in their work. It 
was found that while the programs articulated similar goals and sensibilities about the role of technology in 
achieving them, their approaches to process, conceptions of outcomes, and future visions for using technology 
varied. Ultimately, the findings may offer relevant indicators for how these technologies may expand, shape, and 
affect some of the most critical efforts in the field of international education now and in the future. 

4.1 IVC Can Extend the Reach of Cross-Cnltnral Programming 

The programs expressed common goals, which mirror those of the broader international exchange field, but 
which originated from ideas about how IVC and online chat rooms might be used to achieve them. In interviews, 
one of the nonprofit’s program staff described the initial idea behind the program, saying “IVC can be such a bridge, 
because. . .people don’t have a lot of money to do exchanges, so why don’t we do more virtual exchanges and bring 
the world into the classroom. . .” (interview, educational program 1) and the other nonprofit described employing the 
“broadest way to kind of foster cross-cultural understanding” to “develop and support this global diverse community 
who use immediate technologies like IVC to foster cross-cultural understanding and respecf ’ and a “broader online 
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social network intended to engage a diverse audience interested in social change” (interview, educational program 

2 ). 

While one organization expressed the broad goal of “connecting them (students) with these other parts of 
the world” (interview, educational program 1), the other organization staff focused on improving the U.S. /Muslim 
world relationship, saying “it’s not necessarily natural that a young kid in the Bible Belt will connect with a young 
kid in Gaza. . (interview, educational program 2). The third organization staff shared that “post-9/1 1 there was a 
lot of funding from the State Department to foster mutual understanding. . .between teachers, students and their peers 
in the Muslim world, so we’ve put a concerted effort into ensuring that those relationships are happening” 

(interview, educational program 3). 

In interviews, interactive videoconferencing emerged as an increasingly standard tool in the field of 
international education today, a way to extend the experience and benefits of exchange to new places and people. 
“Who is doing IE without technology?” remarked one program (interview, educational program 3). One of these 
nonprofits sees new media “just as a tool for promoting understanding and facilitating dialogue. . . It encourages 
more collaborative approaches,” but is “truly based on how you use those technologies” including “as a platform for 
engaging people for social change” (interview, educational program 2). One program noted that “especially in 
underperforming schools. . . it’s interactive so it keeps their attention” (interview, educational program 1). 

4.2 Perceptions of the Role and Valne of Facilitation Varied 

The technologies employed by the three programs include live videoconference (used by all three), email, 
chat rooms, and social media. The data most closely examined interactive videoconference and chat rooms use and 
the experience. The research focused on how programs conceive the interactive experiences they support - and what 
emerged as a point of differentiation was how they discussed the fact of and approaches to structuring and 
facilitating the interactions themselves. 

While it often seems there is a sense that new media and technologies themselves are a panacea for 
inclusiveness, the programs expressed that these spaces require attention or they can become, in the words of one of 
the organization staff, “communities like self-echo chambers” (interview, educational program 2). “There’s a lot of 
assumption that new media. . . opens up, as its about diversification and creation about choices. But in actuality, new 
media is used just as a tool.. People are not opening up for diversification and do not discuss information that create 
pre-existing biases. . .How one socializes within one own social network is how people want one to do” (interview, 
educational program 2). 

All the programs, in slightly different ways, connect classrooms around the globe on topics of interest and 
importance to participants - but there was variation in their conceptions of the role of facilitating the interactive 
spaces and sessions. 

Two of the three programs which were interviewed facilitated the interactive, live videoconference sessions 
that comprise the IVC tool used by all of them. One of the staff expressed that they “put of lot of attention into what 
kind of processes we need to follow in order to have new media connection so that people from diverse backgrounds 
can connect and communicate. ..to create a safe space for diverse community. . . dialogue” and sees the “critical role 
of facilitators to encourage fundamentally eager level of engagemenf’ (interview, educational program 2). This role 
serves the program goals by “keeping things on track, making sure different voices are heard,” they described. 

While they admit this may be perceived as an “artificial process” it is instrumental to allow students to separate their 
views from their inner biases” (interview, educational program 2) - a clearly stated goal of the program. 

Another organization’s videoconference sessions take a more hands-off approach, which they describe as 
“moderation” (interview, educational program 1). “It’s not our role to guide the debate,” describes a program staff 
“We’ve never had to stop a videoconference, but some of them have gotten heated,” but “we just sat back and 
listened” (interview, educational program 1). Observations of this organization’s videoconference sessions (live and 
previously recorded) indeed showed groups of students engaging in a prolonged Q & A on a variety of topics 
(genocide, culture, Palestinian-Israeli conflict, etc.) with a moderator in a limited role helping repeat questions that 
were inaudible and telling the groups when it was time to move to the next question. 

However, as observed in the Egypt/US/Canada session, this resulted in frequent rephrasing - as opposed to 
repeating - of questions that may have misconstrued the inquirer’s intent; and the highly managed Q & A format 
often had a “ping-pong effecf ’ - where, though each question was asked and answered effectively, no real or 
sustained dialogue emerged. On occasion, follow-up questions were posed, but more often the response signaled the 
end to exploration of that topic. It was also observed that the Q & A and marginal involvement of the adult 
facilitator frequently resulted in a few most vocal students participation while others looked on (observations, 
educational program 1). 
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Some programs saw technologies used to supplement the live videoconferencing, such as online chat 
rooms, as ideal spaces for facilitating transformative dialogues: “with written communication you have time to 
reflect and de-escalate if needed- it brings out something that live might not if we’re in each other’s faces and we’re 
arguing and we don’t see the topic in the same way, sometimes the online communication can give people time to 
reflect before responding. . . people are really honest when you might never meet them. . .so it allows people to really 
say what they‘re thinking” (interview, educational program 3). This organization expressed that they do not guide or 
facilitate the actual classroom/school interaction, but scan the online chat rooms and see the dialogues progress and 
participants self-police themselves in the interactive space. Its staff notes: 

“(There are) so many examples where a student will post something inflammatory and someone else 
will respond and so a teacher calls attention to it or we see it ourselves - there’s just so few instances 
where we’ve gone in there ourselves and said - ‘children’ . . . denied them access. It’s a community of 
students - so someone will come on and say, ‘I think you’re making generalizations and not all 
people are like this and I’m actually Irish and I’m not like that’ - and another one will come on and 
say, ‘I’m glad you said that because I’m also feeling that’. . .it’s amazing, they self-moderate.” 

(interview, educational program 3). 

4.3 Technology Has Value, But Faces Challenges 

Key benefits to using technology expressed in interviews focused on the value of extending the cross- 
cultural experience to people who might not otherwise have interacted, whether because of geography, politics, 
economics, or students’ own personal challenges. 

One program stated the value as “access, the multiplier effect of using the technology - the cost of bringing 
one exchange student to the US for a year is nearly $18-25,000. That could train - huge number of national 
trainings. And cell phones - I have been everywhere and no one doesn’t have cell phones. I feel like that about 
Internet too - not in each home, but there’s a little kiosk and they’re all gathered around it... So being able to have 
that far reach is completely due to the technology” (interview, education program 3). 

From an international politics perspective, countries or cultures whose relationship in state or diplomatic 
spheres can find channels to interact and normalize relations at least while collaborating. “The fact that someone 
can be on a project involving so many countries. . . to the U.S. /Russian example. You can be working with someone 
from wherever and you don’t have to say we’re partnering with them but we’re part of this environmental project 
and we’re all working together, so sometimes it allows groups that aren’t officially sanctioned to work together to 
still be in communication with each other” (interview, educational program 3). One program actually trains students 
to utilize new media to tell and share what they’ve learned. “A major challenge with young people connecting to 
sensitive issues of Muslims or Jews is that they have to reply to their parents and friend... that is the kind of bravery 
or courage we are trying to encourage. We put them with the skills and tool sets of how you shoot things with video 
camera, “digi” or “Flip” camera? How do you tell stories in a compelling and effective way? And how to get access 
and distribution to truly get these out to the world?” (interview, educational program 2). 

Still, programs agreed that challenges to using IVC abound - both technical and substantive. “It works 80% 
of the time,” says one program, “it can be very slow, sometimes it freezes, it drops off’ (interview, educational 
program 1). “The connection drops, they can’t go online, the computer broke. . . it’s constantly having to manage 
peoples’ expectations” (interview, educational program 3). In observations of both live and previously recorded 
videoconferences, “technical difficulties” required repeating of questions, as well as lagging camera focus so that 
students “cannot tell if there is a reaction” to their questions/responses (field notes, educational program 1). 

While connectivity is increasingly pervasive, capacity and geography still present challenges. Especially 
when working with “poorer communities and rural areas in the U.S. . ..if s just getting them the technology,” while 
abroad, “it depends. . . in India, we have no problems. But certain parts of Africa, very challenging” (interview, 
educational program 1). All programs expressed the challenges of managing translation. 

4.4 JVC’s and Chat Rooms Future in Cross-Cultural Programming 

Beyond technical challenges, imagining the future of these programs and how they employ IVC and online 
chat rooms, programs emphasized expanding reach and longer term impact, translating the online experience to the 
real world. 

One of the educational nonprofits sees their future use of IVC focused on connecting with people who 
haven’t had exposure but are really critical to the community to which they plug in to. “So we work really hard not 
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to just get already globalized young people who are important and great, we love them, but we also try to get who 
haven’t had ability to connect cross-culturally” (interview, educational program 2). 

Another organization articulated interest in connecting the online experience to offline international 
development efforts. “What we're trying to do is make our programs more sustainable. . .make more of our programs 
in the future tied to viable long-term development. . .How does the school work with continuing relationships?” 
(interview, educational program 1). For example, a program with Haiti involved “live link-ups, connecting them to 
school rebuilding projects and carrying that out so that in two years we can say that your funds have built fifty 
schools, look what you've done” (interview, educational program 1). The third organization also agreed that the 
connection between interaction online and action on the ground -behavioral change - is important: “did they go 
back to their communities and start doing community service projects?” (interview, educational program 3). 

Engagement of existing new and social media is also on the minds and in the future plans of these 
programs. One of these nonprofits is experimenting with “taking things to a much different level - from Muslim 
broadcast publisher A1 Jazeera to BBC. . .getting much bigger audience. . . We are kind of trying to leverage 
traditional and social media to have a bigger impact” (interview, educational program 2). But one of these 
organizations admits to struggling in making the argument to experiment in the space. “There’s still convincing: 
people make an association that it’s a pen pal organization or it’s Facebook, or some of our funders say - this 
(online) Collaboration Center we work really hard to run, they say, ‘why don’t you just bring them all onto 
Facebook, use it to connect’ ... So I think it’s still sort of a different model,” (interview, educational program 3) 


Findings 

Educational Program f 

Educational Program 2 

Educational Program 3 

IVC Can Extend 
the Reach of 
Cross-Cultural 
Programming 

IVC is seen as a bridge because 
schools don’t have a lot of 
money to do exchanges, so why 
don’t we do more virtual 
exchanges and bring the world 
into the classroom?! 

IVC and a broader online social 
network are intended to engage a 
diverse audience interested in a 
social change. 

Interactive technology has 
emerged as a way to extend the 
experience and benefits of online 
project-based collaboration to 
new students and places. 

Perceptions of the 
Role and Value of 
Facilitation 
Varied 

The moderator has a limited 
role helping repeat questions 
that were inaudible and telling 
the groups when it was time to 
move to the next question. 

The virtual spaces require 
attention or they can become, in 
the words of the Program 
Director, "communities like self- 
echo chambers." Facilitation has 
instrumental value in helping 
adults separate their views from 
their inner biases. 

Participants are expected to be 
self- moderators. A teacher who 
plays the role of a facilitator does 
not guide the actual 
classroom/school interaction, hut 
scans the online chat rooms and 
sees the dialogues progress and 
participants self-police 
themselves in the interactive 
space. 

Technology has 
value, hut faces 
challenges 

Technical difficulties during a 
live videoconference require 
repeating of questions, as well 
as lagging camera focus 
confuses students with if there 
is a reaction to their 
questions/responses. 

A major challenge with young 
people connecting to sensitive 
issues of Muslims or Jews is that 
they have to reply to their parents 
and friends. The challenge lies in 
empowering youth with shooting 
with video camera so that they 
tell stories in a compelling and 
effective way. 

Several times the connection 
drops, students can't go online or 
the computer broke. It’s 
constantly having to manage 
peoples ’ expectations. IVC is a 
different model which still needs 
its acceptance amongst funders 
who think of it as a pen-pal 
system. 

IVC’s and chat 
room’s Future in 
Cross-Cultural 
Programming 

The program desires to he more 
sustainable by tying online 
experiences of students with 
offline international long-term 
development efforts. 

The program sees its future use 
of IVC focused on connecting 
with people who haven 't had 
exposure hut are really critical to 
the community to which they 
plug in for a bigger attitudinal 
impact. 

The program also aspires to build 
connection between online 
interaction and action on the 
ground as a result of behavioral 
change in student participants. 


5. Discussion 


Paulo Freire’s concepts of democratic, inclusive, learning environments, driven by robust dialogues that 
reflect real world issues and encourage active engagement with them would find much to admire in these initiatives. 
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Exploring how these three programs perceive the value of and different approaches to using IVC and chat room 
helps to shed light on an area of educational technology that is growing and little examined. Key program 
discussion of variance in approach and method may serve to inform future pedagogy and anticipate how and to what 
extent these and emerging interactive technologies and techniques will be adopted in the future, across the field of 
international education. 

5.1 Technology Challenges Are Snrmonntable, bnt Reqnire Approaches Appropriate to tbe Space 

The challenges represented in the data fell into categories related to the technical (mirrored largely in 
existing research), experiential, and sustainable, that is - how does a program extend and/or apply the online 
experience to the offline world. 

Anastasiades suggests that “if one believes that videoconferencing is like conventional face-to-face 
interaction, they will be disappointed” - and students reported in evaluations “frustrations with technological lags” 
(educational program 1, evaluation report). But while programs ran into mechanical and media glitches which 
affected participant experiences, these did not seem to inhibit programs’ overall enthusiasm or sense that the issues 
would be worked out over time. However, certain instances can be problematic when participants are groups already 
sensitized to misunderstanding. In an observation of a one of the organization’s videoconference between US and 
Muslim students, the microphone on a Muslim girl with hijab was not functioning properly and she could not be 
heard - an achingly symbolic example with the potential to undermine the whole idea to dispel stereotypes about 
political and social practices. Further, the moderator was then required to rephrase each response, in effect, speaking 
for her - another unintended problem with larger significance among certain populations. 

5.2 Deliberate Facilitation Can Be Important to Participant Experience 

Existing research suggests that the application, design, and ways that technology is used to determine its 
value (Clark, 2000) and though the use of videoconferencing technology in education is increasing, there is a lack of 
pedagogical design (Newman, 2008). Further, there is a dearth of research on what facilitators do in varied online 
learning environments or how facilitation contributes to interaction and learning (Sheingold, 2005). This research 
attempts to contribute to that conversation. 

The study’s data show that these international education programs are experimenting with and learning 
from different processes and pedagogical approaches to interactive engagement that serve their goals of mutual 
understanding. Their level of sophistication of process was often correlated to their goals. One of the program that 
expressed an interest in outcomes related to attitudinal change among participants, sees the “critical role of 
facilitators to encourage fundamentally eager level of engagemenf ’ and to “create a safe space for diverse 
community. . . dialogue” (interview, educational program 2). This group was the most cognizant of the larger default 
position among educational programs - suggested in the existing literature- to view new and social media 
particularly as a panacea, the tool as strategy whose mere existence is meaningful and effective. This organization, 
instead, articulated that they deliberately construct a conflict resolution model process and employ IVC thaf exhibits 
multiple screens at once, eliminating or mitigating many of the commonly noted challenges of the technology. This 
process is developed to address and achieve specific goals of attitudinal change, so it’s much more intricately 
structured to meet those ends. 

On the other hand, the other organization,, whose approach is “not to guide the debate,” finds its 
participants are often frustrated with the brevity and “ping pong” effect of the videoconference sessions - which 
they construe as less conducive to deeper or sustainable dialogue on any of the issues the sessions aim to cover. 
Observations of these sessions showed a very simplistic structure of interaction that reflected very little deliberate 
shaping for the challenges of the technology. While this organization has much broader goals of connecting students 
- versus the abovementioned organization’s specific aims of attitudinal change - they articulated future intentions 
with IVC and other new media that reflected a greater attention to pedagogy. There was a marked sense of desire to 
insert more of their goals into the online/offline spaces. 

Overall, the organizations expressed that they are finding, to varying degrees, that new media spaces 
require adjustments appropriate for the space - either to better approximate a live experience, to achieve their stated 
states, or to maximize the experience for participants. 
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5.3 Future Focus ou Pedagogy aud Sustaiuability 

The most interesting and potentially valuable issues that the data may eontribute relate to the ehallenge of 
making interaetive spaees meaningful. Refining and adjusting pedagogy for these spaees - stmeturing them to 
extend the impaet after the virtual experienees, will allow for greater sustainability and for programs to reaeh 
populations valuable to aehieving their goals. 

Anderson et al. suggest that effeetive online diseourse must be guided toward higher levels of learning 
through refleetive partieipation, as well as by ehallenging assumptions and diagnosing miseoneeptions. Further to 
the discussion of process, above, it seems that regardless of the character or complexity of their goals, the programs 
that employed any kind of moderator were interested in better managing their processes to fit the online space - and 
to more precisely accomplish their goals. 

Data show programs considering how to connect online experiences with action in the real world. The 
literature suggests that learning spaces function best if embedded in cultural and social context committed to their 
innovation (Punie 2007) - and sustaining and applying the experiences that happen online, to the offline world was 
of interest to all of the programs. One of these organization talked about linking with development efforts for field 
work, and live visits and exchanges that happened as a result of the online experience. The other organization is 
trying to extend their reach to key populations by partnering with social and mass media outlets to engage them in 
their dialogues. 


6. Conclusion 


Those who believe that utilizing information and communication technologies, specifically 
videoconferencing in the classroom, is a tool of the future should take a careful look at the increasing number of 
international education programs that are employing technology as a basis for creating collaborative understanding 
across cultures. This research seeks to add to the current trends of how organizations perceive technology as 
contributing to their goals, and how this can be used in cross-cultural understanding in students. The research reveals 
how different international education programs engage I VC and chat rooms in distinctly different ways in order to 
achieve varying degrees of understanding based on specific programmatic goals. 

Additional research would be need to be conducted in order to grant a clearer understanding of the long-term 
effects, and address techniques most suitable in creating a prevailing impact. The study was illuminating in many 
respects, but begs further research into how organizations will use IVC and facilitation to achieve long-term impacts 
from their programs. The research suggests following up with past participants of such programs, and determining if 
technology assists in achieving goals of the international education program. As well, it will be interesting to see if 
these technologies become a trend with a broader reach into more comers of the world to include more marginalized 
(geographically, economically, socially) populations. 

A progressively shrinking world inevitably identifies the necessity for mutual understanding among global 
citizens. International education programs create a cross-cultural window of dialogue through linkages shaped by 
IVC, which strives to provide mutual understanding in a world where globalization is taking root. There is 
extensive use of information and communication technologies in the international education community when 
striving to achieve their program goals, which indicates an escalating reliance on tools like IVC and online chat 
rooms in the globalizing world. 
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Abstract 

This study examined pre-service English teachers in relations to goal orientation theory of motivation. 
Embedded mixed design was used in the study. That is to say, quantitative data as a primary form was used to 
inform the qualitative phase. In the quantitative part of the study, data were collected on a questionnaire. On the 
basis of responses from 186 pre-service English teachers, qualitative data were collected in form of one-on-one 
interviews to understand the motivation and behavior variables of the pre-service teachers. The results indicated that 
the mastery and performance oriented pre-service teachers displayed different characteristics of motivation. 
However, some of the participant had more than one goal orientation. The results as well as their implications are 
discussed and suggestions for future research are presented. 

Introduction 

Teachers are one of the most significant components in education systems. And one of the main objective 
in education systems is to reach a goal. To achieve this goal, many variables interact with each other. The main 
variable that affects the teachers, the learners, and the teaching and learning environment is achievement goal 
orientation construct. Achievement goal orientation has been pointed to affect the teaching and learning environment 
in numerous studies (Butler, 2007: Coutinho & Neuman, 2008; Elliot & Murayama, 2008). This concept is about 
purposes and motives that individuals had in a task. This variable has been used to assess the students’ motivation 
for learning but later it has begun to be used to understand the teachers’ motivation, as well (Butler, 2007). 
According to Nitsche, Dickhauser, Fasching, and Dresel (2011), teachers’ goal orientation is a significant factor for 
teachers’ individual development of competence. Retelsdorf et ah, (2010) stated that achievement goal construct 
may also be useful for defining motivation of pre-service teachers as well. 

Teachers’ attitudes during education process affect their behaviors in teaching (Bandura, 1997). Recent 
studies done in the field of education have showed that knowledge and skills are not adequate for active teaching. 
Teachers’ attitudes and beliefs have also been found to be contributing to their effectiveness as educators (Bandura, 
1997; Tschannen-Moran, Woolfolk Hoy, & Hoy, 1998). Even though there are many studies on students’ motivation 
for learning, there has been little research on teachers’ motivation for teaching (Retelsdorf, Butler, Sfreblow, & 
Schiefele, 2010). 

So far, there has been very little research on the pre-service English teachers’ goal orientations and in an 
online learning environment. So, this research aimed to explore the pre-service English teachers’ achievement goal 
orientations. 


Literature Review 


Achievement Goal Orientation Theory 

Achievement goal theory is very common in the achievement motivation literature that affects the pre- 
service teachers. This construct is about the purposes and motives the individuals get in achievement task (Dweck & 
Leggett, 1988). It consists of four constructs. These are: mastery-approach, mastery-avoidance, performance- 
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approach, and performance-avoidance achievement goal orientations (Elliot & Murayama, 2008). Learners with 
approach goals try to master learning tasks and they do their best to completely acquire the subjects, while students 
with avoidance goals avoid negative results such as failure while mastering the tasks (Elliot & McGregor, 2001). In 
other words, in approach aspects (mastery-approach and perfonnance-approach) learners believe in themselves to do 
well but in avoidance aspects (mastery-avoidance and perfonnance-avoidance) these learners doubt about their 
ability to perfonn well (Coutinho & Neuman, 2008). Kucsera et al. (2011) reported that achievement goal theory 
affect many motivation and behavior variables in the student and work literatures, but it is still applied specifically 
to teachers and teaching. Nitsche et al. (2011) reported that teachers’ goal orientations are significant factors for 
teachers’ individual development of competence. Retelsdorf et al. (2010) cited that achievement goal construct may 
also be useful for defining motivation of pre-service teachers. 

The literature provides proofs that achievement goal orientations are contributing factors of success of 
learners and pre-service teachers (Bandura, 1997; Coutinho & Neuman, 2008; Elliot & McGregor, 2001). However, 
there is a gap in the literature that identifies achievement goal orientation to pre-service teachers studying in an 
online environment. This study adds evidence and knowledge to the literature. 

Method 


Research Design 

Embedded mixed design, which is one of the mixed method design, was used for this study. The purpose of 
the embedded design is to collect quantitative and qualitative data simultaneously or sequentially, but to have one 
form of data play a supportive role to the other form of data (Cresswell, 2011 :544). In this study a sequential design 
was used with the embedded design. That is to say, quantitative data as a primary form was used to inform the 
qualitative phase. For the quantitative part of the study we collected data on Achievement Goals Questionnaire- 
Revised (AGQ-R) during a month. This quantitative information represented a major source of information for this 
study. For the qualitative part we collected data in the form of one-on-one interviews to understand the beliefs and 
perceptions of the pre-service teachers at first hand. 

Figure 1: The study design 



Participants of the study 

The present study was conducted at the Department of Distance English Language Teacher Education BA 
program at Open Education Faculty, Anadolu University, Turkey. This program has a blended model of instruction, 
supplies both face to face, and online distance education. The first two years of the program are conducted mainly 
through traditional classroom instruction, and the third and the fourth year of the program are conducted by means of 
distance education supported by online courses. 

Simple random sampling as a form of probability sampling was used for the quantitative part of the study. 
This method is often used with a population that has a small number of cases (McMillian, 2004: 109). This approach 
was convenient because the population was small and numbered. The population consisted of pre-service teachers 
enrolled at Distance English Language Teacher Education BA program. Participants were 186 senior students. In the 
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present study, there were 144 females and 42 males students. The age range of the sample was 21 years to 30 years. 
Table 1 provides the demographic information for the population, and for the sample used in this study. As seen in 
Table 1, the proportions of the gender and age within the sample are generally representative of the population. 

Table 1: The Proportions of the Gender and Age Within the Sample and Population. 



Sample 

(n=186) 

Population 

(N=588) 

Gender 

Female 

77.4% 

76.7% 

Male 

22.6% 

23.3% 

Age 

21-23 

53.2% 

56.7% 

24-30 

46.8% 

43.3% 


Qualitative part of the study was consisted of two one-on-one interviews. Participants of the one-on-one 
interviews were 2 pre-service English teachers. The participants were purposefully selected from 186 pre-service 
teachers who completed the Achievement Goals Questionnaire - Revised (AGQ-R). The participants were selected 
for the one-on-one interviews because their scores on the scales identified them as the most representative of one of 
the two groups. The first group has high mastery (low performance) goal orientations while the second group has 
high performance (low mastery) goal orientation. 

Data Collection Instruments 

The data were obtained, primarily, via Achievement Goals Questionnaire - Revised (AGQ-R) (Elliot & 
Murayama, 2008). The participants also responded to the demographic questionnaire. For the qualitative part, data 
were collected through open-ended questions from one-on-one interviews. 

The AGQ-R is a 12 item, 5-point Likert scale questionnaire used to measure achievement goal orientations. 
Participants who scored high on items associated with mastery-approach items indicated that they strive to 
thoroughly learn course materials. The mastery-approach goal is associated with higher performance and academic 
success (Elliot & Murayama, 2008). Participants who scored high on the mastery-avoidance approach items 
indicated that they attempt to master course material to avoid failure and recrimination (Elliot, 1999; Elliot & 
Murayama, 2008). Participants who scored high on performance approach items indicated they strive to master 
course material to outperform their peers. Participants who scored high on performance-avoidance items possess low 
motivation and fear of failure associated with negative outcomes (Elliot & Murayama, 2008). Based on the data 
from the sample used in this research, the Cronbach alpha coefficients of each subscale were; 0.82 for mastery- 
approach, 0.91 for mastery-avoidance, 0.83 for performance-approach and 0.89 for performance-avoidance. The 
AGQ-R’s Cronbach coefficients were relatively high in Elliot & Murayama’s study, too. These Cronbach alphas 
were 0.84, 0.88, 0.92, and 0.94, respectively (Elliot & Murayama, 2008). The demographic information section, 
which was developed by the researchers, includes questions about gender, age, educational background, and 
teaching and learning experience of the pre-service English teachers. 

In qualitative one-on-one interviews, open-ended questions below were asked to the pre-service English 

teachers: 

■ How do you describe yourself as a teacher/student? 

■ What are your personaFacademic/professional goals? 

■ What do you do when you faced a very difficult academic task? 

■ What do you think about the relationship between ability and success? 

Procedure 

A pilot study was conducted before the main qualitative study. The aims of conducting this pilot study were 
to check whether or not the data collection tool worked well, to learn whether an addition to the data collection tool 
was necessary, to find out whether the questions were clear or not and understandable enough . To conduct the pilot 
study, the researchers emailed all the senior students. Out of the 588 students, 38 consented to participate voluntarily 
to the face-to-face pilot study. After the piloting procedure, which lasted for a month, the researchers performed 
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minor revisions regarding to language and grammar of the survey. The survey link was then put on the online 
eourse room and the diseussion board and partieipants were asked to eomplete the survey thoroughly and honestly. 
If the learner agreed to partieipate in the study, he/she eompleted the survey, and then submitted the eompleted the 
one-time survey through the seeure online link. The partieipants voluntarily eonsented to partieipate in the study 
without ineentives. At the end of the survey, whieh stayed for a month in the online eourse room, 186 partieipants 
were attended the study. 

To support the primary quantitative data, supportive qualitative data were added. Out of the 186 
partieipants, 2 pre-serviee teaehers were purposefull seleeted for the one-on-one interviews. The representative pre- 
serviee teaehers were eontaeted by telephone for the interviews and eaeh interview was seheduled for an hour and 
took between 30 to 60 minutes. Content analysis was eondueted in order to analyze the qualitative data obtained 
from the interviews. The answers to eaeh question in the interviews were eoded and eategorized by authors. 

Table 2: Aehievement Goals 



High Performance 

Low Performance 

High Mastery 

High Mastery/ High Performance 

High Mastery /Low Performance 

Low Mastery 

Low Mastery /High Performance 

Low Mastery /Low Performance 

Findings 


Quantitative Data 

Deseriptive statisties for aehievement goal orientations and its subseales are presented in Table 3. The 
reliabilities eomputed for the subseales are also shown in Table 3. 

Table 3: Descriptive Statistics and Cronbach Alpha Measures for Achievement Goal Orientations Scale (n=186) 



M 

Sd 

Min. 

Max. 

Cronbach 

Alpha 

Mastery- Approach 

4.21 

0.93 

1.00 

5.00 

0.82 

Performance-Avoidance 

2.88 

1.44 

1.00 

5.00 

0.91 

Performance-Approach 

3.18 

1.35 

1.00 

5.00 

0.83 

Mastery- Avoidance 

4.02 

1.06 

1.00 

5.00 

0.89 

General AGO 

3.60 

0.93 

1.00 

5.00 

0.89 


According to Table 3, results indicated a difference in the pre-service English teachers’ goal adoptions. 
Even though these results suggest that the pre-service English teachers adopt more than one achievement goal, most 
of the pre-service EEL teachers adopt mastery goals for learning. This result can be interpreted as positive since 
teachers adopting mastery goal orientations seek challenging tasks and do well in difficult situations. However, 
mastery goal orientations involve the development of competence through task mastery and the emphasis is placed 
on developing new skills (Lindsay, 2010). Moreover, the mastery goal orientations are positively related to self- 
efficacy beliefs. 

The findings also suggested that there were not noteworthy findings between pre-service EEL teachers’ 
achievement goal orientations and gender, age and work experience. However, the analyses also revealed no 
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statistically significant relationships between achievement goal orientation, and students’ socio-demographic 
characteristics. Furthermore, the findings showed that nearly all of the pre-service teachers had more than one goal 
orientation. 

Qualitative Data 

Pre-service English Teacher Adopting Mastery Goal Orientations 

How do you describe yourself as a teacher /student? 

The pre-service teacher says “I am a hardworking student and teacher. . . and a quick learner.” She also sees 
herself as a competent learner and therefore she learns efficiently. “I was always a successful student and I believe 
that if you study hard you do wonders.” She emphasized that her aim as a teacher was to learn for learning and this 
is her primary aim. 

What are your personal/academic/professional goals? 

The teacher personal goal is to finish the school on time. She added “After I graduated I have to study 
harder to teach better. . . If I give up studying I couldn’t teach deeply. . . so my goal is to be a perfect teacher.” She 
thought that she is a successful teacher and her grades are proofs for this result. Fler academic and professional goals 
were simple. She wanted to graduate with an honor degree and began to work as an instructor in a public university. 
She added that a teacher should be enthusiastic and wise. That’s why she wants to be an enthusiastic and wise 
teacher when she begins to work. 

What do you do when you faced a very difficult academic task? 

The teacher believes that every student can be successful if he/she studies hard. When she faces a difficult 
task, first she gets the bottom of the subject. If she didn’t understand then she asks her teachers or friends. She 
emphasized that she never leaves the task incomplete. She says “When I learn, I learn completely.” 

What do you think about the relationship between ability and success? 

The teacher believes that one can succeed only when he/she studies hard. “This is a psychological 
situation. . . I believe that ability is nothing without studying. . . If the teacher has an ability he/she needs to study to 
succeed but if he/she has an ability this does not enough for success. . .” 

Pre-service English Teacher Adopting Performance Goal Orientations 

How do you describe yourself as a teacher /student? 

The teacher says “I am not very smart student but I study all my lessons and I do my best in the exams. . . 
Mostly I get average grades ... I am a student like that. . . that’s enough for me.” She adds that “If I didn’t study and 
get low grades I felt guilty. . . I don’t want to let my family down. . . I try hard to make them feel happy.” 

What are your personal/academic/professional goals? 

For academic goal, the teacher responded that she wanted to finish the school on time. “The grade point 
average doesn’t matter for me ... I just want to finish my school and make my father happy and believe in me 
because he doesn’t believe that I will finish the school on time.” As a personal goal, she says “After I graduate, I 
want to study in an elementary school because I love kids a lot. . . And I want to spend all my spare time in my 
house. . . with my children... after I graduate I don’t want to spend my time with challenging situations, exams. . . I 
feel tired of these.” 

What do you do when you faced a very difficult academic task? 

The teacher says “It is hard for me to deal with that tasks. . . Mostly I lose my motivation on that 
occasions. . . so it is better for me not to think too much on that tasks.” She added that she tried to understand and 
overcome it but mostly she chose to get it out of her way. “If I do the simple tasks and have an idea about the 
difficult ones that is enough for me.” 

What do you think about the relationship between ability and success? 

The teacher responded that “ability is very important factor in success. . . for example when I am not good 
at subject or lesson, I know that I cannot do well, so I don’t force myself because I know that I don’t have the 
necessary ability.” She says “I believe that I don’t have much ability in English so I study hard before the exams 
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and mostly I get around 70 points. . . actually it is enough for me. . .some of my friends don’t study hard and they get 
80-90 points. . . because they have the ability for that .” 

The pre-service English teachers describe themselves in terms of goal orientations. The differences exist 
between mastery and performance goal orientations are shown on table 4. As seen in table 4, learning aim, 
motivation, ability, effort, interest in learning and improvement are the characteristics identify the pre-service 
teachers. 

Table 4: Characteristics of the Goal Orientations 


Characteristics 

Mastery 

Performance 

Learning aim 

Competence, self-improve 

Avoid failure 

Motivation 

Intrinsic 

Extrinsic 

Ability 

Can be developed 

Innate 

Effort 

To learn 

Look-good 

Interest in learning 

High 

Low 

Improvement 

Choose challenging tasks 

Choose less challenging 
tasks 


Discussions and Recommendations 

The main purpose of this study was to describe the achievement goal orientations of the pre-service English 
teachers. The quantitative and subsequent qualitative data and analysis provided consistent results. In literature, 
studies about goal orientations found different thoughts and behaviors between mastery and performance goal 
orientations. In this study, the mastery and performance oriented teachers displayed different characteristics, too. 

The results in this study also supported the previous studies (Lindsay, 2010; Nitsche et al. 2011; Kucsera et al.2011; 
Pudelko & Boon, 2014). However, some of the participant had more than one goal orientation. These participants 
expected to be very successful as their motivations and achievement behaviors are high (Dweck, 1998; Yeung, Tay, 
Hui, Lin, & Low, 2014). In qualitative part, the themes namely learning aim, motivation, ability, effort, interest in 
learning and improvement led to differences when mastery and perfonnance orientations compared. The learning 
aim and motivation came up as a powerful determiners in adopting goal orientations. 

Further studies should be replicated with a bigger population to better assess and evaluate achievement goal 
orientation. In addition, qualitative studies in a long term may give different and deep results. Mastery and 
perfonnance goals need further interpretation with male and female balanced studies. Besides, the four quadrants of 
goal orientations with a bigger group need to be studied. And more research is needed about how these orientations 
affect the students of these teachers adopting different goal orientations. 
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Abstract 

This paper reports the findings of a follow-up study of eleven students who graduated in 2010 from the online MS in 
Instructional Technology (WebIT) program at the University of Tennessee. In the fall of 2013, these students 
completed an online follow-up survey to share perceptions of their program experiences regarding program 
attributes and post-program experiences. The nature of their program experience feedback from the follow-up 
survey was consistent with feedback given three years earlier in response to a program completion survey. Their 
comments were generally positive with few specific suggestions for program improvement and indicate consistency 
regarding their perceptions of the value of selected online program characteristics. The follow-up survey included 
several new questions regarding impact on student career progression. Discussion of results from the follow-up 
survey will be organized using the Community of Inquiry Model to illustrate the relevancy of this theoretical 
framework for classifying issues related to successful program design. 

Perceptions of Online Program Graduates: A 3 -Year Follow-Up Study 

In the spring of 2010, eleven students successfully completed an online, 2-year Master’s Degree program 
(WebIT) in Instructional Technology (IT) at The University of Tennessee at Knoxville (UTK). In the fall of 2013, 
these eleven students were asked to complete an online survey to share their program experiences and suggestions 
for program improvement. All eleven students agreed to complete the survey. This paper will describe the results of 
an analysis of the survey responses from these eleven program graduates. Though this is a small case study, the 
context of the case reflects the present state of elective student matriculation in new, online programs in higher 
education. The results of this analysis may be of use to others designing and implementing online instructional 
programs in similar contexts. 


Background 

Twenty-five students were recruited for the first cohort of the WebIT program in the spring semester of 
2008. Only eleven of the students completed the cohort at the end of the spring semester 2010, an attrition rate of 
56% (Waugh, DeMaria, & Trovinger, 2010). The WebIT program was organized using a cohort model in order to 
provide the students with an opportunity to benefit from the potential of increased social engagement with a 
consistent group of classmates. This organization was planned as a means to increase the likelihood of the 
development of a Collaborative Learning Community, or academic Community of Practice (Hung & Chen, 2001), 
among the students. 


Literature Review 

Commuuity of luquiry Model. A major challenge for online education that has been cited in numerous 
prior studies (e.g., Daves & Roberts, 2010; Mayes, Luebeck, Ku, Akarasriwom, & Korkmaz, 2011; Moore, 2013; 
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Palloff & Pratt, 2007), is students’ lack of sense of community in virtual space. It is not uncommon that the physical 
separation among students in an online learning environment can result in online learners feeling isolated, 
disconnected and disengaged (Palloff & Pratt, 2007). This isolation can become a major cause of student 
dissatisfaction and may lead to increased program attrition. The Community of Inquiry (Col) model was first 
introduced by Garrison, Anderson, and Archer (2000) to provide an explanation for how this lack of physical 
presence, connectedness and interaction might affect the emergence of an Online Learning Environment (OLE). The 
Col theory posits that communication and interaction in online learning environments is crucial to the development 
of higher order thinking skills among online learners. The model itself provides a framework for classifying the 
important interactional patterns in online learning environments. The Col model consists of three main elements: 
social presence, cognitive presence, and teaching presence (see Figure 1). These three elements overlap with one 
another to some extent, while each of them characterizes a distinct type of human interaction in an online learning 
environment. 

Teaching presence, an essential element of the Col model, directly impacts the quality of the educational 
experience. This element consists of three characteristics: design and administration, facilitating discourse, and 
direct instruction (Anderson, Rourke, Garrison, & Archer, 2001). Teaching presence and instructor roles in OLEs 
determine online learning satisfaction and success in the sense that they significantly influence how students engage 
in deep learning in an online setting (Garrison, Cleveland-Innes, & Fung, 2010; Villagran-Glover, 2012). Mayes, 
Luebeck, Ku, Akarasriwom, and Korkmaz (2011) synthesize instructor roles into five major categories: the 
manager, the technical advisor, the facilitator, the social director, and the educationalist. 

Social presence is the element of the Col model that represents the set of social interactions that enable 
participants to create a viable community. Social presence is what allows independent individual activities to 
become part of a collaborative process. This element encompasses three characteristics: affective communication, 
open communication, and group cohesion (Garrison, 201 1). Projecting an effective social presence in a virtual 
environment, can help online learners successfully collaborate with each other in order to more effectively learn 
together. 


Cognitive presence is defined as “the extent to which the participants in any particular configuration of a 
community of inquiry are able to construct meaning through sustained communication” (Garrison et ah, 2000, p. 
89). This is the key element of Col. Garrison and Cleveland-Innes (2005) provide valuable guidelines for creating 
and sustaining cognitive presence in an effective OLE. The cognitive presence component of the Col model 
represents students' ability to construct meaning through communication. 


Community of Inquiry 



Communication Medium 

Figure 1. Community of Inquiry Model (reprinted from Garrison, Anderson, & Archer, 2000, p. 88) 


189 





In summary, Garrison et al. (2000) emphasize that in order to build a Community of Inquiry, an effective 
Online Learning Environment, instructors should establish an educational experience by employing neither a 
traditional "sage on the stage" nor a "guide on the side" approach. The goal should be to shift the traditional roles of 
students and instructors, and to highlight group dynamics, that is, to let all participants interact and contribute to the 
learning as a member of the community. Garrison et al. (2010) revisited and tested the Col model using a survey 
instrument based on the Col conceptual framework. The results of their study confirmed that the Col model is a 
valid and valuable theoretical tool to help online educators understand the complexities of the essential interacting 
factors of teaching, social and cognitive presence in online education settings. Swan and Ice (2010) compiled a 
number of Col studies conducted during the last decade after the emergence of the Col model, and concluded that 
the development of the Col model has made major contributions in moving online education forward. Garrison and 
Akyol (2013) drew the same conclusion in their review of a series of studies on Col. In the present study, most of 
the comments shared by the WebIT students also can be classified according to one or more of the three main 
elements of the Col model. 


Method 

The gathering of long-term follow-up data from program participants is critical for understanding the 
lasting impact that instructional programs can have on program participants (Ruspini, 2002). Data gathered long 
after the graduates might have any concerns regarding possible negative consequences from their honest 
assessments, provides another perspective in evaluating the suitability of program attributes and characteristics. 
Though similar data are often gathered in other contexts, to date, no other studies have been found that report the 
long-term perceptions of online education graduates regarding their program experiences. Such data are important 
because they can inform program designers about the longer-term value of online curricula. Additionally, follow-up 
data provide an opportunity for program graduates to provide details regarding the degree to which intended 
program outcomes match with current employment requirements. 

This follow-up study was conducted as part of a larger case study (Stake, 1995) to determine the viability 
of the WebIT curriculum model. The follow-up data collected for this study took place during the fall semester of 
2013, three years after the students graduated from the program. The follow-up survey sought to address the 
following broad research questions: (a) How did the graduates perceive selected online program attributes? and (b) 
What recommendations would they offer regarding program format, curriculum pacing, student workload, and 
instructional design? 


Limitations 

The data from this follow-up study provide a small, non-random view of the specific circumstances 
perceived by the participants in the first cohort of the WebIT online program. The perceptions of these students are 
typical insofar as they represent an actual sample of students who were attracted to an online program leading to a 
Master’s degree in Instructional Technology at a state university in the southeastern United States. The range of 
ideas expressed represent one set of all possible student feedback. The proportions associated with response patterns 
are simply one possible sample and may or may not reflect a true proportion of student perceptions in similar 
contexts. While these are significant limitations, the data reported are a “real” characterization of the responses of 
typical students to one specific online learning environment and as such illustrate a snapshot of student concerns 
about the characteristics and effectiveness of an online learning environment. 

Selected Findings 

The WebIT 3-year follow-up survey instrument included a total of 18 items, 13 of which were open-ended 
questions (see Table 1). Selected findings and discussions are presented in this section. Based on the survey results, 
six thematic categories of student perceptions of WebIT emerged: (1) benefits of online learning, (2) professional 
impact and development, (3) teaching presence, (4) social presence, (5) cognitive presence, and (6) program design. 

A number of the students’ follow-up survey responses were highly consistent with their program 
completion survey responses from three years earlier (Waugh & Su, 2011). The 2010 WebIT Online Program 
Completion Survey asked for these graduates’ feedback to the WebIT curriculum designers, instructors and IT 
faculty. Both the 2010 and the 2013 surveys obtained information regarding student perceptions of their 
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expectations and actual online program experiences, although the survey questions were worded slightly differently 
in the two surveys. The 2013 survey specifically asked about how their program experiences prepared them for their 
current employment (See Table 1). The student responses to both of these surveys indicated a high degree of 
satisfaction with the WebIT experience. In particular, students reported a high degree of satisfaction with the cohort 
program structure and the collaborative components of the OLE. However, they also proposed recommendations for 
future WebIT program improvement, such as clearer, more detailed activity/assignment instructions, and a higher 
level of faculty involvement. 


Table 1 

The WebIT 3-Year Follow-Up Survey 


1 : What is your gender? [f/m] 

2: What is your approximate age? [20-30/31-40/41-50/51-60/61 years old or older] 

3 : Please tell us about your current employment situation and whether or not this has changed since you completed 
the WebIT program. 

4: Did the WebIT program help to prepare you for your current work responsibilities? [n/y] 

5: [If respondent answered “y" to Q4] How did the WebIT program prepare you for your current work 
responsibilities? 

6: How, in your opinion, could the WebIT program have beher prepared you for your current work responsibilities? 

7: Please tell us about any professional development activities (enrollment in academic degree programs, 
attendance/participation at professional conferences, participation in job-related training, etc.) you have engaged in 
since completing the WebIT program in 2010. 

8: Please describe any characteristics of the WebIT program that you found to be of value to you. 

9: Please describe any characteristics of the WebIT program that you feel could be improved to better meet your 
personal needs. 

10: Do you have any recommendations for how WebIT might be improved to better meet your professional needs? 

1 1 : Would you recommend a program like the WebIT program to other students with similar professional interests? 
[n/y] 

12: Having experienced both online and face-to-face academic program format, which do you prefer? Why? 

13: Would you enroll in another online academic degree program? [n/y] 

14: [If respondent answered “v" to 0131 Whv would vou enroll in another online academic program? What 
characteristics of online programs make them desirable to you? 

15: [If respondent answered “n ” to 013] Whv wouldn't vou enroll in another online academic program? What 
characteristics of online programs make them less than desirable to you? 

16: What academic program characteristics, i.e., how a program is organized and delivered, do you feel best meet 
your needs as a student? 

17: Please tell us about your interactions with other members of your WebIT cohort since completing the WebIT 
program in 2010. In what ways have you communicated with or continued to be involved with the other members of 
the cohort? 

18: Please share any additional thoughts you may have about your experiences as a student in the WebIT program. 
Tell us anything you would like for us to know about your experiences in WebIT. 


Benefits of Online Learning. A majority of the students (10 out of 1 1) identified the most critical aspect of 
online format of the WebIT program to be its flexibility and convenience in accommodating their academic needs 
and time constraints. This theme was identified through their responses to Q8, Q12, and Q14. This finding is 
consistent with those reported by other researchers regarding online learners’ common perceptions of the benefits 
associated with online learning (Carr, 2000; Chen, 2013; Mayes et al., 2011; Moore, 2013). Mayes et al. (2011) 
summarize many of these important strengths of online learning as “a powerful and previously unavailable 
professional development alternative” for educators and learners who are financially or geographically restricted. A 
number of comments illustrate the importance of flexibility and convenience to these students. 

Response to Q8: The most important thing for me was the fact that it was online. 

Response to Q12: 1 like the flexibility we had with the online format, we could work on our own time and 

find ways around conflicting schedules to get together and work collaboratively. 

Response to Q14: convenience is the most important reason I would enroll in an online program [^ic]. 

Response to Q14: The rigor of this program The convenience for the working individual [s'ic]. 
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Response to Q14: When done right they [online programs] can bring great advantages, such as better time 
management & time flexibility (you can find the best time for you to work on assignments, study, take 
tests, etc - also don't have to fight traffic and try to find parking around campus) [^;c]. 

Response to Q14: Flexibility. On demand. Less travel [^;c]. 

Response to Q14: The time factor, mostly — I think if I had an infinite amount of time and attention. I'd 
ultimately prefer face-to-face classes, but with the realities of adult life, online classes are more appealing: 
even though the actual workload may be a little more strenuous, the benefit of not having to travel and 
being able to work on my own timeframe (to some extent) far outweigh the extra work [^;c]. 

Professional Impact and Development. Q3 and Q5 asked about students’ employment situations and any 
perceived relationships between their participation in the WebIT program and their current employment. About half 
of the students (5 out of 1 1) reported that the WebIT program had a positive impact on their technology knowledge 
and job performance. A majority (9 out of 10) reported that since completing the WebIT program, they were 
engaged in additional professional development activities such as professional conferences and in-service training 
sessions, many of which were electronically-mediated. In response to Q7, they linked their post-graduation 
participation in such activities to their engagement in similar activities during the WebIT program. One specific 
recommendation, in response to Q18, was for the program to integrate a professional internship experience. Several 
students provided the following elaborative comments: 

Response to Q3 : 1 have become a lead teacher since completing the program. 

Response to Q3: 1 am the STEM teacher at my school which is Grades 4-7. This position was created after 
the WebIT course and I by far was the most qualified for it [s'ic]. 

Response to Q3 : Just recently I have been named the multimedia center manager and. Score that I was able 
to change my position. The web it program was indeed the reason I was able to go further [^;c]. 

Response to Q5: 1 believe that the program enabled me to become a more technologically advanced teacher 
which [^;c] brought me to the attention of the administration who asked me to become a lead teacher. 
Response to Q5: Participating in the WebIT program gave me numerous skills I, otherwise, wouldn't have 
gotten with ease. It prepared me for the majority of the career positions I hold now. It's playing a key role in 
continuing to reach for other career goals [^;c]. 

Response to Q7: 1 have attended the TETC conference twice. 

Response to Q18: There could be a sort of internship class where for 1 semester the students are required to 
volunteer with a school or company and do a tangible and relevant project and present their progress and 
completed product to the class or something [^;c]. 

Teaching Presence. When asked about the characteristics of the WebIT program that the students felt 
could be improved to better meet their personal needs (Q9), 6 of the 1 1 students mentioned that some aspect of 
course organization or program design could be improved. These comments specified the need for more course 
organization, use of rubrics, less constructivist and more direct instruction strategies. Two other students criticized 
the WebIT program because it did not provide enough full-time faculty involvement in the teaching of the courses, 
relying instead upon Graduate Teaching Assistants (GTAs) as the primary instructors for specific courses. These 
two students’ comments illustrated their struggle with this issue. 

Response to Q9: Assignments given need to have clear, concise rubrics for grading [s'ic]. 

Response to Q9: 1 was a little disappointed that almost all our classes were led by grad assistants. I would 
have liked to have had more interaction with the professors. This is not to say that the assistants did not do 
a good job [^ic]. 

Response to Q9: 1 feel I could have learned so much more if the actual professors had been actively 
involved in the program. The GTA's were a great bunch of people, helped us a lot and everything, but I 
think I missed out on the professors teaching us and sharing their experiences with us . . . [^;c]. 

Both instructors and students are key participants in building a successful online educational process. 
Garrison et al. (2000) stated that the teacher takes the primary responsibility for performing two general functions in 
teaching presence. Although the WebIT faculty seem to have fulfdled the first function, the design of the online 
educational experience, some students perceived that the faculty neglected the second function, facilitation. These 
students felt that the use of GTAs as primary instructors was not as desirable as using professors in these teaching 
and support roles. These student comments indicate that the multiple instructor roles (Mayes et al., 2011) were not 
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performed to the satisfaction of all students, which impacted the perceived quality of these students’ experiences to 
some extent. The Col framework emphasizes that for creating effective OLEs, instructors must play all their 
simultaneous roles (Swan, Richardson, Ice, Garrison, Cleveland-Innes, & Arbaugh, 2008). Additionally, for the 
student’s experience to be optimal, the student must perceive that this is the case. This finding also begs the question 
of how students might perceive differences among the different classifications of “faculty”. If GTAs are not 
perceived by students as fully qualified teaching faculty (in some sense) then how might students perceive the 
qualifications of part-time or adjunct faculty? This characteristic of an online program — student perceptions of 
faculty quality — might well be linked to overall student satisfaction and program retention. 

Social Presence. The element of Social Presence is reflected by the fact that half of the students (5 out of 
10) responded to Q8 that they found the cohort program organization model valuable in helping them to complete 
the program. In this instance one student shared no comment and one other student mentioned valuing how the 
instructors handled the cohort, but the comment did not convey quite the same sense of personal valuing as the other 
5 student comments. However, 6 out of 9 who responded to the question made some mention of their perceived 
value of the cohort model. 

Response to Q8: 1 enjoyed my cohorts. You really got to know people in the cohort [^;c]. 

Response to Q8: 1 enjoyed the cohort, online classes fit my schedule, flexibility [^;c]. 

Response to Q8: The collaboration among "cohort-mates" - it was encouraging to know there were more 
people struggling just like I was ;) and also knowing we understood what we were going through and how 
we could help each other [^;c]. 

Response to Q8: Not having to leave home for classes and forming a close cohort [s'ic]. 

Response to Q8: The online factor, of course — not having to travel for classes was a huge plus; I liked the 
"cohort" aspect — working with the same people from class to class, and having a definite timeline mapped 
out [s'ic]. 

Response to Q8: 1 really valued the intimate way the instructors and directors handled the cohort, classes, 
processes and interactions. 

A majority of the students (10 out of 11) responded to Q17 that, since completing the WebIT program, they 
stayed in touch with one another via electronic means. These responses demonstrate that, even after graduation, the 
students shared a strong sense of group identification and affinity; and valued what Lehman & Concei9ao (2010) 
describe as “being there” and “being together”. A number of studies (e.g., Daves & Roberts, 2010; Kim, Kwon, & 
Cho, 2011; Swan et ah, 2008; Walton, Cohen, Cwir, & Spencer, 2012) have examined how social connectedness 
greatly affects the nature of learning. These studies report similar findings about the perception of interpersonal 
connections with online peers serving as an important indicator in online learning success. According to Palloff and 
Pratt (2007, p. 237) "it is the relationships and interactions among people through which knowledge is primarily 
generated". Understanding the value of online social presence, can help online learners effectively and successfully 
collaborate with each other (Garrison et ah, 2000; Gayol, 2010). 

Response to Q 17: 1 have had personal conversations with one of my cohorts and have had random 
interactions on social media with 3-4 others. I attended a conference at one cohorts workplace [s'ic]. 
Response to Q17: 1 am sorry to say that I have talked very little to any of my cohort members. I keep up 
with a few of them on facebook but that is about it [s'ic]. 

Response to Q17: Communicate with my two closest classmates a few times a year via email 
Response to Q17: 1 have had limited but regular contact with other cohort members. I have seen them at 
conferences and even presented with one at a conference after our graduation. 

Cognitive Presence. Garrison, Anderson, and Archer (2000, 2001) consider cognitive presence as the most 
basic Col element. Cognitive presence is an indicator of learners’ ability to construct and confirm meaning through 
sustained reflection and discourse. The authors (2001) present the Practical Inquiry (PI) model (see Figure 2) that 
operationalizes cognitive presence. The PI model, defined by two axes, consists of four phases of critical inquiry: 
the triggering event, exploration, integration, and resolution. 
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Figure 2. Practical Inquiry Model (reprinted from Garrison, Anderson, & Archer, 2001, p. 9) 

The Cognitive Presence element of the Col model can be seen reflected in the comments from the WebIT 
students reporting that their experiences in WebIT contributed strongly to their post-graduation ability to solve 
technical and other problems on their own. The students’ comments show evidence of the final phase, resolution, 
where learners settle on a solution and test the solution through real-world implementation. Resolution is the most 
demanding of the four phases of practical inquiry. The WebIT student feedback conveys a clear image of their 
successful knowledge -building processes during and after their online learning in WebIT. Their experiences since 
completing the WebIT program reveal that cognitive presence and growth occurred during their program 
experiences and transferred to new contexts beyond program completion. Several WebIT students provided the 
following elaborative comments: 

Response to Q5: When teachers are planning online activities, I have more knowledge of tools, things to 
consider when setting up, and I am better able to help teachers consider how to implement or plan. By 
learning online, I also can consider the end user point of view. 

Response to Q5: I've used many of the technical skills that I learned in the program. I have become a better 
problem solver. 

Response to Q5: Flelp with networking and terminology, ideas for using Internet for education. 

Response to Q5: 1 think it gave me the foundation for good course design, thinking critically, exposing me 
to different ID theories and views, the ADDIE model, etc. 

Response to Q5: Knowing how to prepare new curriculum and working with tons of new computer 
software. Espically [^ic] managing and maintaining that software in our computer labs. 

Response to Q5: Opportunity to explore/try different technology tools; confidence/footing to propose 
different technology solutions to administration 

Response to Q5: The knowledge that I gained from the program allowed me to bE [s'ic] competitive. It also 
allowed me to learn management skills in project management etc. 

Response to Q18: 1 loved this program. It is an experience that I cherish and am very proud of I owe my 
professional success to the program, and in a way I owe a lot of who I am to the program. I learned more 
than just content, I learned how to learn in a way I had never experienced before. I became a critical 
thinker. The program was truly a life changing event for me. 

Program Design. Because the means of communication differ between face-to-face and online learning 
environments, different pedagogical approaches need to be applied in online teaching as compared to face-to-face 
teaching. When answering Q16, five (out of 10) students recommended that the WebIT online program provide very 
clearly structured instructions, mbrics, and assignments in order to support their learning. 

Response to Q16: Projects are outlined at the beginning with a clear rubric that will be used for grading 

[^;c]. 
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Response to Q16: 1 want a class to... have clear objectives have challenging but reasonable assignments 
have a fair grading system provide feedback in a timely manner [^ic]. 

Response to Q16: Online but structured Material available in the beginning so those who work can work on 
it as they need to [^ic]. 

Response to Q16: Clear expectations Clearly defined rubrics [s'ic]. 

Response to Q16: There are clear expectations. 

As Naidu (2013) pointed out, appropriate instructional design is especially important in online educational 
settings since the flexibility and openness feature of OLEs further complicates teaching and learning processes. 
Instructional designers need to be particularly aware of delivery instructions, mbrics, and assignments “in the most 
valid, reliable, equitable, and secure manner” (pp. 270-271). Instructional designers need to ensure that students’ 
learning is optimized through careful consideration of the instructional design attributes incorporated into the OLE. 
Some students will likely value course and programmatic structural elements to a greater degree than others (Waugh 
& Su, 2011), but this aspect of an OLE is critical to a student’s perception of success. 

Conclusion 

The 3-year follow-up survey responses from the WebIT cohort graduates provide data about their 
perceptions and views of the program characteristics of the WebIT program, and their experiences in the program. 
This paper demonstrates how the WebIT student perceptions and experiences reaffirm the importance of the critical 
elements of the Community of Inquiry model. 

The results of this follow-up study may be of some benefit to administrators and program designers who 
seek to understand more about potential online students and online program design. The data from the study show 
that the WebIT students prefer a program format that is flexible and convenient; and minimizes experiences that 
require specific time constraints. The students felt that the WebIT program had a positive impact on their attainment 
of new skills and knowledge, and that this was demonstrated by successful on-the-job application. The students also 
shared the view that some aspects of the WebIT program design could be improved. Specifically, students would 
prefer more structure within courses to guide independent and group work. Lastly, the students viewed the cohort 
model as a highly valuable organizational model that provided them with many opportunities for meaningful 
collaborative work in pursuit of program goals. 

This follow-up study contributes to a very limited body of literature related to longitudinal perceptions of 
students regarding their online educational experiences. The data from this small case study may not be fully 
generalizable to other contexts, but it is likely that some findings will be common across similar contexts. More 
online programs of study should incorporate a longitudinal evaluation as part of their operation. Such evaluations 
can provide meaningful data to guide decisions regarding program revision and ongoing program delivery. 
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Abstract 

This paper showcases how slight modifications to course design helped reduce student procrastination and 
increase effectiveness of their time management practices in an online, primarily self-directed, capstone course. In 
this course, a learning contract was required to be developed by the students to help them better plan and control 
their learning activities throughout an entire course. Multiple virtual residency sessions were scheduled to help 
students solve problems in the beginning weeks of the course so that they were able to move on to the next steps in 
completing their assignments. The course was designed to require a minimum of instructor intervention and greater 
flexibility for students to determine how and when they would complete assignments. 

Context 

An online advanced (capstone) instructional design and technology (IDT) course was transformed from a 
classroom-based laboratory course to an online platform. The design assumptions were based on theoretical 
evidence that adult learners who are nearing the end of their study programs look for learning experiences that are 
related to their workplace and minimally disruptive of their time (GAEL, 2010). Adult learning theory (Knowles, 

1 986) suggests that learners will identify the work they need to do and plan appropriately to manage their learning. 

Learners were expected to create a lesson and materials for three supporting activities using three different 
technologies (Mindtools) to engage their students in deep learning of subject matter. The format of the course was 
primarily self-directed, with three synchronous virtual sessions within the first few weeks, and a rich set of 
scaffolding resources. The course menu was designed to visibly present activities and resources in an order that was 
helpful in moving learners forward, at their own pacing. Readings, tutorials, self-assessments, project guidelines and 
examples, and grading rubrics were provided to help learners absorb the subject matter and complete assignments 
with confidence that they met course expectations. 

Learners were prompted to develop schedules, choose assignment topics and tools, and determine which 
resources they would use throughout the course. They were required to attend three virtual sessions and submit their 
projects by the last day of the course, or earlier. The course was designed to engage learners deeply in course 
content, provide them with authentic and meaningful design experiences resulting in deliverables they can use in the 
workplace, and facilitate them in developing project management skills and confidence to apply IDT competencies. 
(Souid, Wu, Hall, & Koszalka, in press). 


Academic Procrastination 

Academic procrastination, or needless delays in completing academic assignments, tends to happen when 
students lack time management competencies (Solomon & Rothblum, 1984). Time management is an important 
factor that influences academic grades in traditional educational settings (Trueman & Hartley, 1996). Research 
indicates that a negative relationship exists between academic procrastination and academic performance and 
achievement (Michinov et al., 2011; Rotenstein, Davis, & Tatum, 2009). Online procrastinators tend to perform worse 
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than non-procrastinators in traditional face-to-face classroom (Tuckman, 2005). 

In online courses, which are generally designed with less supervision and more self-directed and 
learner-centered activities, time management thus becomes essential to students identifying sufficient time to 
complete course work and avoiding procrastination. Although the flexibility of an online self-directed learning 
course can engage adult learners in developing their own learning progress to suit their schedules, good time 
management skills is necessary to be successful (Kerr, Rynearson, & Kerr, 2006). 

Few investigations have focused on how course structures can be designed to help students better manage 
their time, reflect often on their learning, and complete self-directed online courses. This study presents ideas based 
on evidence collected from two offerings of an online IDT course that included a variety of time management 
scaffolds. 


Literature Review 

Sloan-C (1997) suggested that leamer-centeredness and project-based learning activities can be key 
elements for effective online learning. These basic recommendations for online learning, however, were not 
accompanied with an explanation of how to successfully engage students in completing course activities and 
achieving learning outcomes. One approach to scaffolding learning activities in online learning can be seen in the 
work of Greene & Land (2000). They experimented with instructional scaffolds to support student learning by 
developing guided questions to assist students in accomplishing their projects. Another approach was to use learning 
contracts to help students define how they would make progress in completing courses. Learning contracts, 
generally a work plan, are created by students and given to the instructor to define how and when coursework will 
be completed. Learning contracts are considered effective tools for students to help them personalize and manage 
their own self-directed learning given their learning and life priorities (Hiemstra & Sisco, 1990). 

In the two iterations of this online course, which provided a flexible work schedule, it was evident that 
students who created learning contracts did not follow them. They often waited until the end of the course to submit 
all projects, rather than following their plan to complete and submit projects intermittently along the way. 

Subsequent project reviews and the final course evaluation suggested that most students condenses a semester worth 
of work into a short period of time (a few days of crunch time) at the end of the course to complete their project 
work; they procrastinated until the end of the semester. Reigeluth (2012) suggested that attainment-based progress is 
one of the core ideas that help with maximize learning. Comparing attainment to time-based progress, Reigeluth 
posited that students tend to move forward when they achieve rather than when a certain amount of time has passed. 
The crunch-time approach of the students thus may suggest that they did not maximize their learning, rather rushed 
to meet the goals of submitting projects by the last day of the course. 

Research Question 

While course deliverables were satisfactory in the end, it appeared that learners procrastinated until the last 
moment, rushing to complete and post assignments. Hence, learners may have developed assignments without 
taking enough time to reflect on and learn about what they were producing. Thus, the question is, what course 
design features might be used in online courses to help learners better use the flexible timelines available to reduce 
procrastination, engage more deeply in content, and increase focus on attainment over time? 

Design Decisions 

This required graduate-level capstone course was specifically designed for students to "take the initiative, 
with or without the help of others, in diagnosing their learning needs, formulating learning goals, identifying human 
and materials resources for learning, choosing and implementing appropriate learning strategies, and evaluating 
learning outcomes” (Knowles, 1975, p. 18). One of the most significant issues noted in observations and evaluation 
data from earlier versions of this course was that students appeared to have insufficient time management skills; they 
often struggled to get work submitted by the end of the course. Upon review of course interactions, learning 
contracts, and actual submission dates it was discovered that, in general, students did not physically engage with 
provided course resources over the length of the course. Thus, it was not clear if they spent much time reflecting on 
their own learning throughout the course or just took time in the last few days of the course to create and submit 
their projects. 
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Knowles (1986) proposed contract learnings as a tool to help adult learners organize their learning time. 
When adult learners set up their own learning objectives, they will have a better understanding of their preferred 
learning styles so that they are able to customize their learning to better access learning content (Boyer, 2003). 

In this course, learning contract frameworks were provided to scaffold student progress and project 
management. A template for creating a learning contract was provided so that learners could indicate which 
deliverables they would be working on throughout the weeks of the course and when they would plan to submit their 
projects. Given the tenet of flexibility of the course, there were no intermittent due dates for project, rather students 
were required to submit a learning contract where they defined when they would post their projects. They were 
encouraged to review their contracts often and update their progress. Although students created the contracts with 
dates for submissions, they still tended to wait until the end of the semester to start their work, they spent a minimal 
amount of time with the course resources, and they submitted their projects on the last day of the course. 

The initial design of this online course included only one due dates for projects, the last day of the course. 
Learners, however, were able to complete and post their projects, thereby finishing the course, the day after the last 
required virtual session (generally about 3 weeks into the 8 week course). Dedicated time over a 2-3 week period was 
sufficient to complete all projects, if learners had the dedicated time. One synchronous, virtual session was required at 
the beginning of the course to set context for the course and describe the capstone project requirements and course 
resources. Two optional virtual sessions were also offered within the next two weeks to answer questions and review 
progress. No other communication or activities requirements were included to the course so that learner time was 
focused on completing the capstone projects to demonstrate program competence. 

Upon review of student reflective journals, instructor reflective journal, and course formative feedback, it 
was suggested that more online virtual sessions would be helpful to guide students in keeping on track to complete the 
course work. Students requested discussion boards to share among themselves and communicate more often, however 
when provided during the courses they went unused. Therefore, the synchronous virtual sessions design was modified. 

In this current design, three required sessions were included with in the first three weeks of the course. Each 
was re-designed to include a specific purpose requiring learners to share progress on projects. Although a loss of 
flexibility, the session requirements provided initial goals, helped learners manage their time more effectively, and 
helped learners feel more confident about their projects. 

A virtual session was facilitated at the beginning of the course. It provided an introduction to the course and 
engaged learners in a series of short activities to introduce resources and project requirements. The second virtual 
session focused on learners sharing initial drafts of their project, next steps they will take, and questions about the 
projects they were currently developing. The third virtual session encouraged group problem-solving of issues 
encountered in projects and a discussion of the final requirements to complete the course. Time was allocated in all 
sessions to engage learners in discussing concepts from readings, tutorials, and project guidelines, thus encouraging 
learners to spend time in the course materials in preparation for the virtual sessions. Finally, time was also allocated to 
sharing progress and modifications to learning contracts. Such techniques can be helpful in encouraging 
attainment-based progress. 

Learners were also able to contact the instructor about specific questions outside of the virtual sessions. It 
was observed in these course iterations however, that the instructor spent a great deal of time responding individually 
to learners about similar questions. Thus a Frequently Asked Questions (FAQ) section of created for the course to 
respond to logistical, content, and project questions in between or after the virtual sessions. Learners could then access 
the FAQ area before contacting the instructor and waiting for a response. This helped students develop a more 
independent approach to problem solving and become more efficient of their own time. 

Compared to the initial course design, these enhancements in the course structure design enabled students to 
manage their time in shorter intervals. They no longer needed to plan the entire course schedule upfront, which often 
led to ignoring the plan and reacting to dates. 


Research 

A developmental research approach was used to investigate this iterative re-design process. Multiple types 
of data were collected from three primary resources: course data, learning contracts, final learner projects. The 
course data consisted of self-reports of learner pre- and post-thoughts on instruction and learning in this course and 
distance education experiences in general; post-thought on the virtual residency sessions; formative feedback on 
specific aspects of the course, summative course evaluations, and course site statistics. Learning contracts 
documented learners’ proposed learning activities and assignment submission date. The fourth part of the data, final 
learner product critiques, provided information from the project deliverables. 

A majority of the course-related data were quantitative, while data form journals and projects were 
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primarily qualitative. Data were coded and tabulated to identify themes in response to questions. 


Result and Recommendation 

According to a preliminary review of the data on learning contracts it was determined that the learning 
contracts were not very successful in supporting the self-directed learning. All learners enrolled in the course 
submitted their learning contracts on their proposed date. 

• Less than 20% of the proposed learning activities on the contracts were followed. 

• More than 50% of the proposed learning activities were completed more than 10 days later than proposed 

date. 

Learners who submitted assignments on the day they proposed in their learning contract were logged as 
zero. Those who posted projects earlier than their proposed date were logged as positive. Those who posted projects 
later than their proposed date were logged as negative. 

Of the 8 learners. . . 

• one followed the submitted learning contract fairly closely with a value of -0.6, 

• one partially followed the submitted learning contract with a value of -3.7, 

• six did not seem to follow their learning contracts at all with an average value of -27.8. 

Note that there is no data suggesting whether students had finished projects early but did not submit them 
until the end of the semester. Moreover, 3 of 8 learners proposed in their learnings contracts that they would 
complete the optional content self-assessments (quizzes), yet there is no evidence showing that they completed any 
of these optional self-assessment (Hall et al, in press). 

Table 1 summarizes three different learning contracts, proposed by different learners, showing learning 
activities and milestones with proposed dates for completion and submission. See Table 1 . 

In summary. Learner A appeared to follow the learning contract closely, with an average submission of less 
than one day late compared to proposed submission dates. Learner A independently planned and self-directed 
activities throughout the course session and had final products that were well designed, developed, implemented and 
evaluated earning a final grade of A. Learner B appeared to partially follow the learning contract. Learner B’s final 
products were also well designed, developed, implemented, and evaluated resulting in a final grade of A. Evidence 
suggests that Learner B did fairly well in self-directed activities. Learners C did not seem to follow the learning 
contract. The final projects were submitted all at once and after the final due date of the course. The projects were 
not rated highly, resulting in a final grade of B-. It appeared that Learner C did not perform well in a self-directed 
environment. 

The other data from the courses provided feedback about the virtual sessions and learning contracts that 
may provide rationale for the use of learning contracts and other design features that my help learners reduce 
procrastination tendency in self-directed online instruction. For example, during each of the three virtual sessions, 
learners had opportunities to share their projects, describe their progress, and ask questions. After each sessions 
learners completed post session learning experience surveys and at the end of the course completed an evaluation 
that contained questions about the learning contracts and virtual sessions. Learners appeared to agree that the virtual 
sessions were helpful to promote critical thinking, to keep them engaged in learning about the course outcomes, and 
to develop an understanding of their strengths and weaknesses. The virtual sessions also helped students feel 
motivated to explore content related questions that would help them create their projects. However, in the absence of 
the virtual sessions many students seem to begin procrastination activities, as a majority waited until the end of the 
course to post projects. In the evaluations some suggested that they were holding projects until the last due date. 
There seemed to be confusion about the purpose of the learning contract and learners seemed to think about it as a 
necessary activity with little value. . . there did not appear to be a connection to the idea that such a tools could 
support their time management. 

Given the themes that emerged from the data the team is strongly considering eliminating the learning 
contract from the course, extending the length of required virtual residency sessions, engaging learners in more 
review of (practice with) course resources during the virtual sessions, and developing some interim schedules for 
project submissions. Although these prompts may remove some of the flexibility in completing the course, the 
additional structure may help engage the learners in more reflection on each projects. Project feedback will still be 
primarily provided during the virtual sessions where students will be prompted to share their projects while all 
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provide comments. This will provide learners with synchronous feedback and hopeful support ongoing reflection as 
they move forward. These design modifications, will hopefully help learners avoid procrastination and assist them in 
developing confidence in preparing and defending their design decisions. 


Table 1 . Learning Contract Summary 



Proposed 

Final 


Date 

Goal 

Date 

Comment 


May 27 

Submit Learning Contract 

May 27 

Submit on proposed date 


May 27 

Complete Prototype 1 draft 

NA 

no data 

Learner 

A 

Jun 3 

Submit Prototype 1 

June 6 

Submit 3 days late 

Jun 17 

Write/Submit final Eval Report 

June 17 

Submit on proposed date 

Jun 17 

Submit Prototype 2 

June 17 

Submit on proposed date 


Jun 24 

Submit Prototype 3 

June 25 

Submit 1 day late 


Jun 24 

Complete course surveys /evals 

June 25 

Submit 1 day late 


May 27 

Submit Learning Contract 

May 26 

Submit on time 


May 2 7- Jun 1 

Draft Prototype 1 

NA 

no data 


May 27-Jun 1 

Complete self-assessment 

June 20 

completed self-assessments 
did not follow proposed dates 

Learner 

B 

Jun 4-5 

Submit Prototype 1 

June 8 

Submit 3 days late 

Jun 9-13 

Conduct test of Prototype 1 

NA 

no data on draft completion. 

Jun 14-16 

Write/Submit final Eval Report 

June 25 

Submit 9 days late 


Jun 9-16 

Submit Prototype 2 

June 17 

Submit 1 day late 


Jun 21-22 

Submit Prototype 3 

June 29 

Submit 7 days late 


Jun 23 

Complete course surveys and 
evaluation 

June 29 

Submit 6 days late 


May 27 

Submit Learning Contract 

May 27 

Submit on time 


May 27 

Complete Prototype 1 draft 

NA 

no data on draft completion 


May 31 

Complete Prototype 1 

NA 

no data on completion date, 
could have finished earlier 
but not submitted. 


Jun 5 

Complete/Submit Prototype 2 
for comment 

NA 

no data, may have finished 
earlier, not submitted 

Learner 

Jun 15-17 

Conduct test of Prototype 1 

NA 

no data 

C 

Jun 18-19 

Write/Submit final Eval Report 

July 5 

Submit 16 days late, late 
submission, final date 6/30 


Jun 18-20 

Submit Prototype 2 

July 1 

Submit 1 1 days late, late 
submission, final date 6/30 


Jul 1-2 

Submit Prototype 3 

July 1 

Submit on time, late 
submission, final date 6/30 


Jul3 

Complete course surveys and 
evaluation 

July 2 

Submit 1 day earlier, late 
submission, final date 6/30 


Conclusions 

As this course is continually enhanced based on evidence from previous offerings we hope to determine 
better ways to support adult learners in taking charge of their own learning and developing confidence in their thinking 
and products. In the earlier course designs, attempts to help learners progress and use time wisely did not seem to work 
well with a variety of learners in the course. The enhancements in course structure enabled some learners to manage 
their time in short intervals, while others procrastinated. 

Design enhancements in the future offerings will hopefully help us move closer to resolving procrastination 
issues and enhancing confidence building while maintaining the earlier principles of flexibility and self-direction in 
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online instruction that increases learner-centered decisions and management of learning and minimizes instructor time. 
Although learning contracts were thought to be valuable in promoting self-directed learning, in this case most of the 
leaners completed the contract but did not appear to see value in its use. Concentrating on designing more 
learner-focused types of activities (e.g., sharing projects and progress, practice in identifying provided resources of 
value, emphasizing the need for reflecting on learning) may be more valuable in supporting adult learners’ to take 
control and drive their choices in terms of establishing and meeting their own learning goals. 
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